Information communication device

ABSTRACT

An information communication method is provided that includes continuously capturing an image of a subject that transmits a signal by changing luminance, with an image sensor. The method also includes displaying a captured image that includes a box and a subject image for identifying the subject, while continuously updating a position of the subject image on the captured image, according to a positional relationship between the image sensor and the subject established when the image of the subject is captured. The method further includes determining whether the subject image is in the box, and receiving the signal transmitted by the subject when it is determined that the subject image is in the box. In the receiving, information is obtained by demodulating a bright line pattern that appears in a second image data and corresponds to the plurality of exposure lines.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.14/219,137 filed on Mar. 19, 2014 which claims the benefit of U.S.Provisional Patent Application No. 61/907,026 filed on Nov. 21, 2013.The entire disclosure of the above-identified application, including thespecification, drawings, and claims is incorporated herein by referencein its entirety.

FIELD

The present disclosure relates to a method for communication betweenmobile terminals and home electric appliances, the mobile terminalsincluding a smartphone, a tablet terminal, and a mobile phone, the homeelectric appliances including an air-conditioner, a lighting device, anda rice cooker.

BACKGROUND

In recent years, a home-electric-appliance cooperation function has beenintroduced for a home network, with which various home electricappliances are connected to a network by a home energy management system(HEMS) having a function of managing power usage for addressing anenvironmental issue, turning power on/off from outside a house, and thelike, in addition to cooperation of AV home electric appliances byinternet protocol (IP) connection using Ethernet (registered trademark)or wireless local area network (LAN). However, there are home electricappliances whose computational performance is insufficient to have acommunication function and home electric appliances which do not have acommunication function due to a matter of cost.

In order to solve such a problem, Patent Literature (PTL) 1 discloses atechnique of efficiently establishing communication between devicesamong limited optical spatial transmission devices which transmitinformation to free space using light, by performing communication usingplural single color light sources of illumination light.

CITATION LIST Patent Literature

[PTL 1] Japanese Unexamined Patent Application Publication No.2002-290335

SUMMARY Technical Problem

However, a conventional information communication method using light hasa problem that a user of a terminal which receives light is required toperform extra operation since a user interface is insufficient.

In view of this, one non-limiting and exemplary embodiment provides aninformation communication method which achieves improvement in a userinterface and a reduction of a burden on a user.

Solution to Problem

An information communication method according to an aspect of thepresent disclosure is an information communication method for performingvisible light communication, the information communication methodincluding: (a) determining whether conflict over use of a lightreceiving unit which detects light will occur between a first programfor performing visible light communication using the light receivingunit and a second program for performing processing different from thevisible light communication, using the light receiving unit; (b)notifying a user of the light receiving unit that the conflict willoccur if it is determined that the conflict will occur; (c) permittingone of the first program and the second program to use the lightreceiving unit in response to an operation by the user; and (d) causing,by the first program, a computer to execute visible light communicationaccording to light detected by the light receiving unit if the firstprogram is permitted to use the light receiving unit.

It should be noted that these general and specific aspects may beimplemented using a system, a method, an integrated circuit, a computerprogram, a computer-readable recording medium such as a CD-ROM, or anycombination of systems, methods, integrated circuits, computer programs,and recording media.

Additional benefits and advantages of the disclosed embodiments will beapparent from the specification and the drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and the drawings which need not all beprovided in order to obtain one or more of the benefits and/oradvantages.

Advantageous Effects

The present disclosure achieves an information communication methodwhich can improve a user interface and reduce burden on a user.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the disclosure willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the present disclosure.

FIG. 1 illustrates a system configuration according to Embodiment 1.

FIG. 2 is a flowchart illustrating processing operation of a terminalwhich activates a program according to Embodiment 1.

FIG. 3 illustrates an example of light-receiving-unit use informationaccording to Embodiment 1.

FIG. 4 illustrates an example of a list of programs using a lightreceiving unit stored in a program-using-light-receiving-unitinformation store unit according to Embodiment 1.

FIG. 5 illustrates an example of light-receiving-unit conflict useinformation according to Embodiment 1.

FIG. 6 illustrates an example of a notification image displayed on adisplay unit, based on light-receiving-unit conflict use information,according to Embodiment 1.

FIG. 7 is a flowchart illustrating detailed processing operation in stepSC0112 illustrated in FIG. 2 according to Embodiment 1.

FIG. 8 illustrates an example of the light-receiving-unit useinformation of a program according to Variation 1 of Embodiment 1.

FIG. 9 illustrates an example of a list of programs using a lightreceiving unit according to Variation 1 of Embodiment 1.

FIG. 10 illustrates an example of light-receiving-unit conflict useinformation according to Variation 1 of Embodiment 1.

FIG. 11 illustrates an example of a notification image according toVariation 1 of Embodiment 1.

FIG. 12 is a flowchart illustrating an example of processing operationby a terminal in which plural light receiving units can besimultaneously used to activate a light-receiving-unit use programaccording to Variation 2 of Embodiment 1.

FIG. 13 illustrates an example of a light receiving unit list stored bya light reception control unit according to Variation 2 of Embodiment 1.

FIG. 14 illustrates an example of a notification image displayed in stepSC0211 illustrated in FIG. 12, according to Variation 2 of Embodiment 1.

FIG. 15 is a flowchart illustrating detailed processing operation ofstep SC0213 illustrated in FIG. 12, according to Variation 2 ofEmbodiment 1.

FIG. 16 is a flowchart illustrating processing operation by a terminalaccording to Variation 3 of Embodiment 1.

FIG. 17 illustrates an example of a notification image displayed on adisplay unit according to Variation 3 of Embodiment 1.

FIG. 18 illustrates an example of use of a receiver according toEmbodiment 2.

FIG. 19 illustrates an example of use of a receiver according toEmbodiment 2.

FIG. 20 illustrates an example of an image obtained by capturing animage.

FIG. 21 illustrates a state when a mark enters a box.

FIG. 22 illustrates another example of use of a receiver according toEmbodiment 2.

FIG. 23 illustrates another example of use of a receiver according toEmbodiment 2.

FIG. 24 illustrates another example of an image obtained by capturing animage.

FIG. 25 illustrates a state when a mark enters a box.

FIG. 26 is a flowchart illustrating processing operation of a terminalaccording to Embodiment 2.

FIG. 27 illustrates a system configuration according to Embodiment 3.

FIG. 28 is a flowchart illustrating processing operation by a terminalto receive visible light of visible light communication, and notify auser of a data-usable application.

FIG. 29 illustrates an example of use according to Embodiment 3.

FIG. 30 illustrates an example of decoded data according to Embodiment3.

FIG. 31 illustrates an example of a data-usable application listaccording to Embodiment 3.

FIG. 32 illustrates an example of information (installed applicationlist) stored in an installed application DB according to Embodiment 3.

FIG. 33 illustrates an example of a list of data-usable applicationsdisplayed on a display unit according to Embodiment 3.

FIG. 34 is a flowchart illustrating processing operation by a server tosearch for an application for which decoded data is usable, according toEmbodiment 3.

FIG. 35 illustrates an example of information stored in adata-application association DB according to Embodiment 3.

FIG. 36 illustrates a system configuration according to a variation ofEmbodiment 3.

FIG. 37 is a flowchart illustrating processing operation by a terminalto receive visible light of visible light communication, and notify auser of a data-usable application, according to the variation ofEmbodiment 3.

FIG. 38 illustrates an example of a data-usable application listaccording to the variation of Embodiment 3.

FIG. 39 illustrates an example of information (installed applicationlist) stored in an installed application DB according to the variationof Embodiment 3.

FIG. 40 illustrates an example of a list of installable applicationsdisplayed on a display unit according to the variation of Embodiment 3.

FIG. 41 illustrates an example of information stored in adata-application association DB according to the variation of Embodiment3.

FIG. 42 illustrates a system configuration according to Embodiment 4.

FIG. 43 is a flowchart illustrating processing operation by a mobileterminal according to Embodiment 4.

FIG. 44 is a flowchart illustrating display processing operation of amobile terminal.

FIG. 45 is a flowchart illustrating processing operation of an IDassociated information conversion server performed when receiving aninformation obtaining request from a mobile terminal according toEmbodiment 4.

FIG. 46 is a flowchart illustrating processing operation by a mobileterminal to detect a user gesture operation according to Embodiment 4.

FIG. 47 is a diagram for describing operation of selecting a visiblelight ID by a mobile terminal B0101 in response to a gesture operationaccording to Embodiment 4.

FIG. 48 illustrates an example of processing operation of selectingdetailed information displayed in response to a gesture operationaccording to Embodiment 4.

FIG. 49 illustrates an example of a gesture-mode association tablestored by a received ID determination unit according to Embodiment 4.

FIG. 50 illustrates an example of an ID handling table stored by thereceived ID determination unit according to Embodiment 4.

FIG. 51 illustrates an example of an ID associated information tablestored by an ID associated information data store unit according toEmbodiment 4.

FIG. 52 is a diagram for describing processing of determining a lightreception mode by the orientation of a mobile terminal according toEmbodiment 4.

FIG. 53A is a flowchart of an information communication method accordingto an aspect of the present disclosure.

FIG. 53B is a block diagram illustrating an information communicationdevice according to an aspect of the present disclosure.

FIG. 54 illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 55 illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 56 illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57A illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57B illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57C illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57D illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57E illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57F illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57G illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57H illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 57I illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 58 illustrates an example of an observation method of luminance ofa light emitting unit in Embodiment 5.

FIG. 59 illustrates an example of each mode of a receiver in Embodiment5.

FIG. 60 illustrates a service provision system using the receptionmethod described in any of the foregoing embodiments.

FIG. 61 is a flowchart illustrating flow of service provision.

FIG. 62 is a flowchart illustrating service provision in anotherexample.

FIG. 63 is a flowchart illustrating service provision in anotherexample.

DESCRIPTION OF EMBODIMENTS

In visible light communication performed using a terminal such as asmartphone, data is received using a light receiving unit such as acamera included in the terminal. Here, a conceivable case is that sinceit is not necessary to show a user of the terminal an image captured bythe light receiving unit, the image captured by the light receiving unitis not shown to the user, and a visible light communication program iscaused to run in the background. It should be noted that a program forperforming processing using a light receiving unit is hereinafterreferred to as a light-receiving-unit use program. A visible lightcommunication program is a type of the light-receiving-unit use program,and is a program for performing visible light communication using alight receiving unit.

Here, if another light-receiving-unit use program (a shootingapplication program, for example) is activated later while the visiblelight communication program is caused to run in the background, conflictover use of the light receiving unit occurs between those programs. Atthis time, the light-receiving-unit use program activated later cannotreserve the light receiving unit already reserved by the visible lightcommunication program, and thus may be forced to terminate.Specifically, if a user activates later another light-receiving-unit useprogram that he/she desires to use without being aware of the visiblelight communication program running in the background, that otherlight-receiving-unit use program is forced to terminate. Thus, the userthinks that the terminal has something wrong due to an unknown cause,and performs various operations on the terminal. In this manner,according to a conventional information communication method, a userinterface is insufficient, which causes the user to perform extraoperations.

In order to address such problems, an information communication methodaccording to an aspect of the present disclosure is an informationcommunication method for performing visible light communication, theinformation communication method including: (a) determining whetherconflict over use of a light receiving unit which detects light willoccur between a first program for performing visible light communicationusing the light receiving unit and a second program for performingprocessing different from the visible light communication, using thelight receiving unit; (b) notifying a user of the light receiving unitthat the conflict will occur if it is determined that the conflict willoccur; (c) permitting one of the first program and the second program touse the light receiving unit in response to an operation by the user;and (d) causing, by the first program, a computer to execute visiblelight communication according to light detected by the light receivingunit if the first program is permitted to use the light receiving unit.

For example, the first program is a visible light communication program,and the second program is a light-receiving-unit use program. In thismanner, if conflict over use of the light receiving unit will occur, theuser of the light receiving unit is notified that the conflict willoccur, and a user operation permits one of the programs to use the lightreceiving unit, and thus the user can understand with ease the status ina terminal which includes the light receiving unit. Furthermore, theuser understands the status thereof, and then can manipulate theterminal. Consequently, conflict over use of the light receiving unitcan be easily solved without extra operation. As a result, improvementin a user interface can be achieved and burden on a user can be reduced.

In step (b), a selection screen for selecting one of the first programand the second program which use the light receiving unit may be furtherdisplayed, and in step (c), if one of the first program and the secondprogram is selected in response to an operation on the selection screenby the user, the selected one of the first program and the secondprogram may be permitted to use the light receiving unit.

In this manner, the user can select a program to be permitted to use thelight receiving unit by his/her operation, and thus a desired one of theprograms is allowed to use the light receiving unit, irrespective of theorder of activating plural programs which use the light receiving unit.In addition, in order to allow a program activated later to use thelight receiving unit, a user does not need to intentionally performoperation for terminating a program activated earlier and using thelight receiving unit. Thus, user operation can be further reduced.

It should be noted that the present disclosure may be an informationproviding method as below. In other words, this information providingmethod is for providing a display unit with information with which auser can select a program permitted to use a light receiving unit if aprogram which uses the light receiving unit is activated in a statewhere another program which has reserved the light receiving unit isrunning, the information providing method including: an activation stepof activating a program for which an instruction is received from theuser; a determination step of determining whether a program which isusing the light receiving unit is present; a reservation step ofreserving the light receiving unit if the light receiving unit can beused; a record step of making a record indicating that the lightreceiving unit has been reserved; a conflict information creation stepof creating conflict use information of the light receiving unit to bepresented to the user if the light receiving unit cannot be reserved; adisplay step of displaying the created conflict information; anexecution program update step of activating a program selected by theuser from the displayed conflict information and terminating a programnot selected.

In step (b), the selection screen may be displayed which includesinformation indicating whether the first program runs in background andwhether the second program runs in background.

This notifies the user whether programs that are to use the lightreceiving unit run in the background, and thus the user allows anappropriate program to use the light receiving unit.

In step (c), another light receiving unit may be assigned to one of thefirst program and the second program in response to the operation by theuser, and in step (d), the first program may cause the computer toexecute visible light communication according to light detected by theother light receiving unit if the other light receiving unit is assignedto the first program.

This permits the first and second programs each to use a different oneof the light receiving unit or the other light receiving unit, thusallowing the programs to be running simultaneously without terminatingone of the programs. As a result, visible light communication andprocessing different from the visible light communication can beexecuted in parallel, thereby achieving a reduction in processing time.

In step (d), data may be obtained by executing the visible lightcommunication, and the information communication method may furtherinclude: (e) displaying an application selection screen for selectingone of plural application programs for which the obtained data isusable; and (f) if one of the plural application programs is selected inresponse to an operation on the application selection screen by theuser, causing, by the selected one of the application programs, thecomputer to execute processing performed using the data.

This allows an appropriate application program to process data obtainedthrough visible light communication.

The information communication method may further include: (g)continuously capturing an image of a subject which transmits a signal bychanging luminance, with an image sensor which is the light receivingunit; (h) displaying a captured image which includes a box and a subjectimage for identifying the subject, while continuously updating aposition of the subject image on the captured image, according to apositional relationship between the image sensor and the subjectestablished when the image of the subject is captured; and (i)determining whether the subject image is in the box, wherein in step(d), when it is determined that the subject image is in the box, thefirst program may cause the computer to receive the signal transmittedby the subject.

In this manner, even if plural subjects which transmit signals bychanging luminance are present being gathered, by merely putting asubject image of a desired object in a box, a signal only from theobject can be easily received.

The information communication method may further include: (j) capturingimages of plural subjects each of which transmits a signal by changingluminance, with an image sensor which is the light receiving unit; (k)displaying a captured image which includes the images of the pluralsubjects and is obtained in step (j); and (l) determining whether aterminal device which includes the image sensor has been shaken, whereinin step (d), the first program may cause the computer to receive thesignals transmitted by the plural subjects, and in step (k), aninformation notification image generated according to a signal from oneof the plural subjects among the signals transmitted by the pluralsubjects and received, and showing information regarding the subject issuperimposed on the captured image in a state where the informationnotification image may be associated with the image of the subject, andif it is determined that the terminal device has been shaken, anotherinformation notification image generated according to a signal fromanother one of the plural subjects which is in a direction in which theterminal device has been shaken, and showing information regarding theother subject may be superimposed on the captured image in a state wherethe other information notification image is associated with the image ofthe other subject.

In this manner, for example, an information notification image having aballoon in which information regarding a subject (detailed information)is stated is displayed in association with the subject, and thus a usercan understand information regarding the subject appropriately. Inaddition, a subject whose information notification image is to bedisplayed can be easily switched by shaking a terminal device.Furthermore, such a subject can be switched to another subject in adirection in which the terminal device has been shaken, and thus theuser can make the switch intuitively, and a user interface can furtherbe improved.

In step (k), for an image of each of one or more remaining subjectsother than the image of the one or the other one of the plural subjectsthat is associated with the information notification image or the otherinformation notification image, an information-presentable notificationimage for notifying that information regarding the remaining subject isfurther presentable may be superimposed on the captured image.

In this manner, for an image of a subject for which an informationnotification image (another information notification image) is notdisplayed, for example, a data-presentable notification image having asmall balloon shape in which detailed information is not described isdisplayed, and thus a user can determine that information regarding thesubject (detailed information) can be displayed as an informationnotification image, by shaking a terminal device.

The information communication method may further include (m) obtainingfirst image data by performing capturing with a first exposure time sothat exposure starts sequentially for a plurality of exposure lines inthe image sensor each at a different time and each of the plurality ofexposure lines partially overlaps in exposure time an adjacent one ofthe plurality of exposure lines, wherein in step (d), second image datamay be obtained by performing capturing with a second exposure timeshorter than the first exposure time so that exposure startssequentially for the plurality of exposure lines each at a differenttime and each of the plurality of exposure lines partially overlaps inexposure time an adjacent one of the plurality of exposure lines, andinformation may be obtained by demodulating a bright line pattern thatappears in the second image data and corresponds to the plurality ofexposure lines, and in step (d), the bright line pattern may be causedto appear in the second image data by setting the second exposure timeto be less than or equal to 1/480 second.

In this manner, as described in Embodiment 5 below, setting a secondexposure time to be 1/480 seconds or less causes an appropriate brightline pattern to appear in image data, and allows high-speed signaltransmission.

The information communication method may further include: (n) performingprocessing of receiving a request to distribute information from arequester; (o) distributing the requested information as associatedinformation associated with a signal received through the visible lightcommunication performed by the first program; and (p) performinginformation processing for charging the requester according to thedistribution of the information.

In this manner, as described in Embodiment 6 below, appropriate chargingprocessing can be performed when providing information through visiblelight communication.

It should be noted that these general and specific aspects may beimplemented using a system, a method, an integrated circuit, a computerprogram, a computer-readable recording medium such as a CD-ROM, or anycombination of systems, methods, integrated circuits, computer programsor recording media.

The following specifically describes embodiments, with reference to thedrawings.

The embodiments described below each show a general or specific example.The numerical values, shapes, materials, constituent elements, thearrangement and connection of the constituent elements, steps, theprocessing order of the steps, and the like described in the followingembodiments are mere examples, and thus are not intended to limit thepresent disclosure. Therefore, among the constituent elements in thefollowing exemplary embodiments, constituent elements not recited in anyof the independent claims defining the most generic part of theinventive concept are described as arbitrary constituent elements.

Embodiment 1

An information communication method according to the present embodimentis a method for allowing a user to select a program permitted to use alight receiving unit if conflict over use of a light receiving unitoccurs due to activating another light-receiving-unit use program in astate where a visible light communication program that is a type of alight-receiving-unit use program has already activated.

FIG. 1 illustrates a system configuration according to the presentembodiment.

A data transmission light C0100 transmits data using light. In otherwords, a data transmission light is a lighting device which performsvisible light communication and transmits a signal by changingluminance.

A terminal C0200 includes a hardware unit C0203, a platform unit C0202,and an application unit C0201. The hardware unit C0203 includes: a lightreceiving unit C0203 a which receives (detects) light emitted by thedata transmission light C0100 and furthermore, can change the shutterspeed (exposure time) when receiving light; a display unit C0203 b whichshows information to a user; and an input unit C0203 c which receivesinput from the user. The terminal C0200 may include a plurality of thelight receiving units C0203 a. The platform unit C0202 includes: a lightreception control unit C0202 a having a function of controlling thestart and end of light reception by the light receiving unit C0203 a andthe shutter speed of the light receiving unit C0203 a; a display controlunit C0202 b which controls the display unit C0203 b; an input controlunit C0202 c which controls the input unit C0203 c; a program controlunit C0202 d which controls a program that the application unit C0201has; and a program-using-light-receiving-unit information store unitC0202 e which stores information on a program that is using the lightreceiving unit C0203 a.

The application unit C0201 has a visible light communication programC0201 a, and a light-receiving-unit use program C0201 b different fromthe visible light communication program C0201 a. The visible lightcommunication program C0201 a has a function of changing the shutterspeed of the light receiving unit C0203 a, using the light receptioncontrol unit C0202 a. In addition, the visible light communicationprogram C0201 a has a function of obtaining data according to lightemitted by the data transmission light C0100, from the light receivingunit C0203 a via the light reception control unit C0202 a. In addition,the visible light communication program C0201 a may have a function of,based on data obtained from the data transmission light C0100, making aninquiry to a Web server and obtaining further additional informationfrom the Web server. In addition, the terminal C0200 may include aplurality of the visible light communication programs C0201 a and aplurality of the light-receiving-unit use programs C0201 b.

FIG. 2 is a flowchart illustrating processing operation of the terminalC0200 which activates a program.

First, the input unit C0203 c of the terminal C0200 receives aninstruction to activate the visible light communication program C0201 aor the light-receiving-unit use program C0201 b (activation instructionor program activation instruction) (step SC0102). The program controlunit C0202 d receives an activation instruction from a user andactivates the program for which the activation instruction has beenissued, via the input control unit C0202 c (step SC0103). Next, theprogram control unit C0202 d determines whether the activated programuses the light receiving unit C0203 a, based on information indicatingwhether the light receiving unit is to be used, which is included inlight-receiving-unit use information of the program (step SC0104).

FIG. 3 illustrates an example of light-receiving-unit use information.

As illustrated in (a) of FIG. 3, light-receiving-unit use information ofa program includes: a program ID_ID0501; a program name C0502 which is aname of the program identified by the program ID_ID0501; informationC0503 which indicates whether the program uses a light receiving unit(whether the light receiving unit is to be used); and a used lightreceiving unit ID_C0504 for identifying a light receiving unit used bythe program. In addition, as illustrated in (b) of FIG. 3,light-receiving-unit use information of another program includes: aprogram ID_ID0505; a program name C0506 which is a name of the programidentified using the program ID_ID0505; information C0507 whichindicates whether the program uses a light receiving unit (whether alight receiving unit is to be used); and a used light receiving unitID_C0508 for identifying the light receiving unit to be used by theprogram.

It should be noted that the program IDs (C0501, C0505) illustrated inFIG. 3 may be each represented using one or more numbers only, one ormore characters only, one or more symbols only, a combination of one ormore numbers and one or more symbols, a combination of one or morecharacters and one or more symbols, or a combination of one or morenumbers, one or more characters, and one or more symbols, rather than acombination of characters and numbers. In addition, the program names(C0502, C0506) illustrated in FIG. 3 may be each represented using oneor more characters only, a combination of one or more characters and oneor more numbers, one or more numbers only, one or more symbols only, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols. Information (C0503, C0507) which indicates whether a lightreceiving unit is to be used illustrated in FIG. 3 may be representedusing one or more numbers only, one or more symbols only, a combinationof one or more characters and one or more numbers, a combination of oneor more numbers and one or more symbols, a combination of one or morecharacters and one or more symbols, or a combination of one or morenumbers, one or more characters, and one or more symbols, rather thancharacters. A used light receiving unit ID (C0504, C0508) illustrated inFIG. 3 may be represented using one or more characters only, one or moresymbols only, a combination of one or more characters and one or morenumbers, a combination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than a number.

In step SC0104 illustrated in FIG. 2, if it is determined that theactivated program is to use the light receiving unit C0203 a, or inother words, the light-receiving-unit use information of the activatedprogram clearly indicates that a light receiving unit is to be used (YESin step SC0104), the program control unit C0202 d checks a list ofprograms using a light receiving unit stored by theprogram-using-light-receiving-unit information store unit C0202 e, inorder to determine whether there is any program currently using thelight receiving unit C0203 a, or in other words, in order to determinewhether conflict over use of the light receiving unit C0203 a will occur(step SC0105). Based on the check result, the program control unit C0202d determines whether conflict over use of the light receiving unit C0203a will occur due to the visible light communication program C0201 a forperforming visible light communication using the light receiving unitC0203 a which detects light and the light-receiving-unit use programC0201 b for performing, using the light receiving unit C0203 a,processing different from visible light communication (step SC0106).Here, if it is determined that the conflict does not occur, or in otherwords, if it is determined that there is no program using the lightreceiving unit C0203 a (YES in step SC0106), the program control unitC0202 d reserves the light receiving unit C0203 a via the lightreception control unit C0202 a (step SC0107). After that, the programcontrol unit C0202 d registers information on the activated program intothe list of programs using a light receiving unit stored in theprogram-using-light-receiving-unit information store unit C0202 e.

FIG. 4 illustrates an example of a list of programs using a lightreceiving unit stored in the program-using-light-receiving-unitinformation store unit C0202 e.

As illustrated in FIG. 4, the list of programs using a light receivingunit includes: a program ID_C1701 for identifying a program using thelight receiving unit C0203 a; a program name C1702 which is a name ofthe program; and a used light receiving unit ID_C1703 for identifyingthe light receiving unit C0203 a being used. It should be noted that theprogram ID_C1701 illustrated in FIG. 4 may be represented using one ormore numbers only, one or more characters only, one or more symbolsonly, a combination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than a combination of numbers and characters. Inaddition, the program name C1702 illustrated in FIG. 4 may berepresented using one or more numbers only, one or more characters only,one or more symbols only, a combination of one or more numbers and oneor more symbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than a combination ofnumbers and characters. In addition, the used light receiving unitID_C1703 illustrated in FIG. 4 may be represented using one or morecharacters only, one or more symbols only, a combination of one or morenumbers and one or more characters, a combination of one or more numbersand one or more symbols, or a combination of one or more numbers, one ormore characters, and one or more symbols, rather than a number.

If it is determined in step SC0106 illustrated in FIG. 2 that conflictwill occur, or in other words, if a program using the light receivingunit C0203 a is registered in the list of programs using a lightreceiving unit (NO in step SC0106), the program control unit C0202 dcreates light-receiving-unit conflict use information, using the list ofprograms using a light receiving unit (step SC0109). Furthermore, theprogram control unit C0202 d displays a notification image based on thelight-receiving-unit conflict use information, on the display unit C0203b via the display control unit C0202 b (step SC0110).

FIG. 5 illustrates an example of light-receiving-unit conflict useinformation.

As illustrated in FIG. 5, light-receiving-unit conflict use informationincludes, for each program that is to use the light receiving unit C0203a, a program ID_C0301 of the program, a program name C0302 of theprogram, information C0303 indicating use status of the light receivingunit C0203 a that is used by the program, and a used light receivingunit ID_C0304 for identifying the light receiving unit C0203 a to beused by the program. It should be noted that the program ID_C0301illustrated in FIG. 5 may be represented using one or more numbers only,one or more characters only, one or more symbols only, a combination ofone or more numbers and one or more symbols, a combination of one ormore characters and one or more symbols, or a combination of one or morenumbers, one or more characters, and one or more symbols, rather than acombination of numbers and characters. In addition, the program nameC0302 illustrated in FIG. 5 may be represented using one or more numbersonly, one or more characters only, one or more symbols only, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than a combination of numbers and characters. Inaddition, the information C0303 indicating the use status of a lightreceiving unit illustrated in FIG. 5 may be represented using one ormore numbers only, one or more symbols only, a combination of one ormore numbers and one or more characters, a combination of one or morenumbers and one or more symbols, a combination of one or more charactersand one or more symbols, or a combination of one or more numbers, one ormore characters, and one or more symbols, rather than characters. Inaddition, the used light receiving unit ID C0304 illustrated in FIG. 5may be represented using one or more characters only, one or moresymbols only, a combination of one or more numbers and one or morecharacters, a combination of one or more numbers and one or moresymbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than a number.

FIG. 6 illustrates an example of a notification image displayed on thedisplay unit C0203 b, based on light-receiving-unit conflict useinformation.

The display unit C0203 b displays, as a notification image, an imageC0401 for allowing a user to select a program which is to use a lightreceiving unit, since plural programs (application programs) use thelight receiving unit (for example, camera). The display of such anotification image notifies a user of the terminal C0200 which includesthe light receiving unit C0203 a, that conflict will occur. In addition,this notification image serves as a selection screen for selecting oneof the visible light communication program C0201 a and thelight-receiving-unit use program C0201 b which are to use the lightreceiving unit C0203 a.

After step SC0110 illustrated in FIG. 2, if a user inputs the result ofselecting one of the programs which uses the light receiving unit to theinput unit C0203 c (YES in step SC0111), the program control unit C0202d receives the result of user selection via the input control unit C0202c, terminates the program not selected, and reserves the light receivingunit for the selected program (step SC01112).

FIG. 7 is a flowchart illustrating detailed processing operation in stepSC0112 illustrated in FIG. 2.

The program control unit C0202 d first determines whether a programactivated later is selected by a user (step SC0112 b). Here, if theprogram control unit C0202 d determines that a program activated earlieris selected (NO in step SC0112 b), the program control unit C0202 dterminates the program activated later (step SC0112 h). On the otherhand, if the program control unit C0202 d determines that the programactivated later is selected (YES in step SC0112 b), the program controlunit C0202 d releases the light receiving unit C0203 a reserved by theprogram activated earlier (step SC0112 c), and clears the list ofprograms using a light receiving unit stored by theprogram-using-light-receiving-unit information store unit C0202 e (stepSC0112 d). Then, the program control unit C0202 d terminates the programnot selected (step SC0112 e). Then, the program control unit C0202 dreserves the light receiving unit C0203 a via the light receptioncontrol unit C0202 a (step SC0112 f). Finally, the program control unitC0202 d registers the program selected by the user into the list ofprograms using a light receiving unit stored in theprogram-using-light-receiving-unit information store unit C0202 e (stepSC0112 g).

It should be noted that in step SC0111 illustrated in FIG. 2, if thevisible light communication program C0201 a is selected, the programcontrol unit C0202 d permits the visible light communication programC0201 a to use the light receiving unit C0203 a in step SC0112. Thevisible light communication program C0201 a permitted to use the lightreceiving unit C0203 a causes a computer of the terminal C0200 toexecute visible light communication according to light detected by thelight receiving unit C0203 a reserved by the program itself.

According to the present embodiment as described above, if conflict overuse of the light receiving unit C0203 a will occur, a user of the lightreceiving unit C0203 a is notified of the occurrence of the conflict,and a user operation permits one of the programs to use the lightreceiving unit C0203 a. Thus, the user can understand with ease thestatus in the terminal C0200 which includes the light receiving unitC0203 a. Furthermore, the user understands the status thereof, and thencan manipulate the terminal C0200. Consequently, conflict over use ofthe light receiving unit C0203 a can be easily solved without extraoperation. As a result, improvement in a user interface can be achievedand burden on a user can be reduced.

Variation 1

An information communication method according to this variation is for auser to select a program permitted to use a light receiving unit, usinginformation indicating whether a light-receiving-unit use program runsin the background or foreground if conflict over use of a lightreceiving unit will occur. It should be noted that a repeateddescription of the drawings already described in Embodiment 1 isavoided.

FIG. 8 illustrates an example of the light-receiving-unit useinformation of a program according to this variation. It should be notedthat the light-receiving-unit use information is used to determinewhether to use the light receiving unit C0203 a in step SC0104 of theflowchart illustrated in FIG. 2, as well as the light-receiving-unit useinformation illustrated in FIG. 3.

As illustrated in (a) and (b) of FIG. 8, the light-receiving-unit useinformation according to this variation includes information indicatingan operation mode (C0605, C0610), unlike the light-receiving-unit useinformation illustrated in FIG. 3. Information indicating this operationmode indicates that a focused light-receiving-unit use program runs inthe foreground that is a mode in which a screen according to the programis always displayed on the display unit C0203 b or in the backgroundthat is a mode in which such a screen is not displayed on the displayunit C0203 b (the user of the terminal C0200 cannot see the screen).

It should be noted that program IDs (C0601, C0606) illustrated in FIG. 8may be each represented using one or more numbers only, one or morecharacters only, one or more symbols only, a combination of one or morenumbers and one or more symbols, a combination of one or more charactersand one or more symbols, or a combination of one or more numbers, one ormore characters, and one or more symbols, rather than a combination ofcharacters and numbers. In addition, program names (C0602, C0607)illustrated in FIG. 8 may be each represented using one or morecharacters only, a combination of one or more characters and one or morenumbers, one or more numbers only, one or more symbols only, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols. Information (C0603, C0608) which indicates whether thelight receiving unit is to be used illustrated in FIG. 8 may berepresented using one or more numbers only, one or more symbols only, acombination of one or more characters and one or more numbers, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than characters. Used light receiving unit IDs(C0604, C0609) illustrated in FIG. 8 may be each represented using oneor more characters only, one or more symbols only, a combination of oneor more characters and one or more numbers, a combination of one or morenumbers and one or more symbols, a combination of one or more charactersand one or more symbols, or a combination of one or more numbers, one ormore characters, and one or more symbols, rather than a number. Inaddition, information (C0605, C0610) indicating an operation modeillustrated in FIG. 8 may be represented using one or more numbers, oneor more symbols, a combination of one or more numbers and one or morecharacters, a combination of one or more numbers and one or moresymbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than characters.

FIG. 9 illustrates an example of a list of programs using a lightreceiving unit according to this variation.

In step SC0108 of the flowchart illustrated in FIG. 2, the programcontrol unit C0202 d according to this variation registers the list ofprograms using a light receiving unit illustrated in FIG. 9 into theprogram-using-light-receiving-unit information store unit C0202 e, usingthe light-receiving-unit use information in the format illustrated inFIG. 8.

The list of programs using a light receiving unit according to thisvariation includes information C0704 indicating an operation mode,unlike the list of programs using a light receiving unit illustrated inFIG. 4. The information C0704 indicating an operation mode indicateswhether a light-receiving-unit use program which reserves the lightreceiving unit C0203 a runs in the foreground that is a mode in which ascreen according to the program is always displayed on the display unitC0203 b or runs in the background that is a mode in which such a screenis not displayed on the display unit C0203 b (a user of the terminalC0200 cannot see the screen).

It should be noted that a program ID_C0701 illustrated in FIG. 9 may berepresented using one or more numbers only, one or more characters only,one or more symbols only, a combination of one or more numbers and oneor more symbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than a combination ofcharacters and numbers. In addition, a program name C0702 illustrated inFIG. 9 may be represented using one or more numbers only, one or moresymbols only, a combination of one or more numbers and one or moresymbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than characters only or acombination of characters and numbers. A used light receiving unitID_C0703 illustrated in FIG. 9 may be represented using one or morecharacters only, one or more symbols only, a combination of one or morecharacters and one or more numbers, a combination of one or more numbersand one or more symbols, a combination of one or more characters and oneor more symbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than a number. In addition,information C0704 indicating an operation mode illustrated in FIG. 9 maybe represented using one or more numbers, one or more symbols, acombination of one or more numbers and one or more characters, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than characters.

FIG. 10 illustrates an example of light-receiving-unit conflict useinformation according to this variation.

The program control unit C0202 d according to this variation createslight-receiving-unit conflict use information illustrated in FIG. 10 instep SC0109 of the flowchart illustrated in FIG. 2, using the list ofprograms using a light receiving unit in the format illustrated in FIG.9.

Unlike the light-receiving-unit conflict use information illustrated inFIG. 5, light-receiving-unit conflict use information according to thisvariation includes information C0805 indicating an operation mode. Theinformation C0805 indicating an operation mode indicates whether aprogram which is to concurrently use the light receiving unit C0203 aruns in the foreground or background. It should be noted that a programID_C0801 illustrated in FIG. 10 may be represented using one or morenumbers only, one or more characters only, one or more symbols only, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than a combination of characters and numbers. Inaddition, a program name C0802 illustrated in FIG. 10 may be representedusing one or more numbers only, one or more characters only, one or moresymbols only, a combination of one or more numbers and one or moresymbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than a combination ofnumbers and characters. In addition, information C0803 indicating lightreceiving unit use status illustrated in FIG. 10 may be representedusing one or more numbers only, one or more symbols only, a combinationof one or more numbers and one or more characters, a combination of oneor more numbers and one or more symbols, a combination of one or morecharacters and one or more symbols, or a combination of one or morenumbers, one or more characters, and one or more symbols, rather thancharacters. A used light receiving unit ID_C0804 illustrated in FIG. 10may be represented using one or more characters only, one or moresymbols only, a combination of one or more numbers and one or morecharacters, a combination of one or more numbers and one or moresymbols, a combination of one or more characters and one or moresymbols, or a combination of one or more numbers, one or morecharacters, and one or more symbols, rather than a number. InformationC0805 indicating an operation mode illustrated in FIG. 10 may berepresented using one or more numbers, one or more symbols, acombination of one or more numbers and one or more characters, acombination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than characters.

FIG. 11 illustrates an example of a notification image according to thisvariation.

If the light-receiving-unit conflict use information illustrated in FIG.10 is used, the display unit C0203 b displays a notification image C0901illustrated in FIG. 11 in step SC0110 of the flowchart illustrated inFIG. 2. This notification image C0901 clearly shows a text “operating inbackground” to indicate a light-receiving-unit use program which runs inthe background, unlike the notification image C0401 illustrated in FIG.6. Such a text is clearly shown to indicate that thelight-receiving-unit use program is to operate in the background, whichallows a user to determine which light-receiving-unit use program can beexecuted simultaneously with a program which does not use the lightreceiving unit C0203 a, and then to select which light-receiving-unituse program is to be used. It should be noted that the notificationimage C0901 illustrated in FIG. 11 is an example displayed using thelight-receiving-unit conflict use information illustrated in FIG. 10,and may show whether the program can be executed simultaneously with aprogram which does not use the light receiving unit C0203 a, instead ofshowing whether the program runs in the background. In addition, thenotification image C0901 may show that a light-receiving-unit useprogram which runs in the foreground is to run in the foreground, ratherthan a light-receiving-unit use program which runs in the background. Inaddition, the notification image C0901 may show that alight-receiving-unit use program which runs in the foreground cannot beexecuted simultaneously with another program which does not use thelight receiving unit C0203 a. In addition, in order to notify a userwhether a program can be executed simultaneously with another programwhich does not use the light receiving unit C0203 a, the character colorof a text of a program name, the background color of a portion where theprogram name is written, and giving a mark in the portion where theprogram name is written may be used in addition to a text description.

As described above, in this variation, a user is notified whether theprograms each of which is to use the light receiving unit C0203 aoperate in the background, and thus the user can allow an appropriateprogram to use the light receiving unit C0203.

Variation 2

An information communication method according to this variation is amethod for allowing a user to select which program is permitted to usewhich light receiving unit if conflict over use of a light receivingunit will occur in a terminal in which one or plural programs cansimultaneously use plural light receiving units. It should be noted thata repeated description of the drawings already described in Embodiment 1and Variation 1 thereof is avoided. In addition, a terminal C0200according to this variation includes, for example, two light receivingunits C0203 a, one of the light receiving units C0203 a is disposed onthe front side of the terminal C0200, whereas the other of the lightreceiving units C0203 a is disposed on the back side of the terminalC0200.

FIG. 12 is a flowchart illustrating an example of processing operationby the terminal C0200 in which plural light receiving units can besimultaneously used to activate a light-receiving-unit use program.

The processing operation by the terminal C0200 according to thisvariation differs from the processing operation illustrated in FIG. 2 inthe following points. The different points are that: a light receivingunit to be used is identified from light-receiving-unit use informationof an activated program (step SC0205); it is determined whether acurrently used light receiving unit on a list of programs using a lightreceiving unit stored in a program-using-light-receiving-unitinformation store unit C0202 e and a light receiving unit indicated bylight-receiving-unit use information of a program activated later arethe same (step SC0207); a light receiving unit list is used to createlight-receiving-unit conflict use information, in addition to the listof programs using a light receiving unit (SC0210); and two types ofinput from a user are a selection of, for each of programs, a lightreceiving unit assigned to the program, and an instruction to terminatea program (steps SC0212, SC0213, SC0214).

In step SC0207 in FIG. 12, a program control unit C0202 d determineswhether a currently used light receiving unit (used light receiving unitID) on the list of programs using a light receiving unit stored in theprogram-using-light-receiving-unit information store unit C0202 e and alight receiving unit indicated by the light-receiving-unit useinformation of a program activated later (used light receiving unit ID)are the same. In other words, the program control unit C0202 ddetermines whether conflict over use of a light receiving unit willoccur. Here, if it is determined that the light receiving units are notthe same (YES in step SC0207), or in other words, if it is determinedthat conflict does not occur, a program activated later can use a lightreceiving unit indicated by a used light receiving unit ID included inlight-receiving-unit use information. Thus, the program control unitC0202 d reserves a light receiving unit indicated by the used lightreceiving unit ID included in the light-receiving-unit use information(step SC0208), and registers information on the activated program intothe program-using-light-receiving-unit information store unit C0202 e(step SC0209).

If it is determined that a currently used light receiving unit (usedlight receiving unit ID) on the list of programs using a light receivingunit stored in the program-using-light-receiving-unit information storeunit C0202 e and a light receiving unit indicated inlight-receiving-unit use information of a program activated later (usedlight receiving unit ID) are the same (NO in step SC0207), or in otherwords, if it is determined that conflict will occur, the programactivated later cannot use the light receiving unit indicated by theused light receiving unit ID included in the light-receiving-unit useinformation. Consequently, the program control unit C0202 d createslight-receiving-unit conflict use information, using the light receivingunit list stored by a light reception control unit C0202 a and the listof programs using a light receiving unit registered in theprogram-using-light-receiving-unit information store unit C0202 e (stepSC0210). Furthermore, the program control unit C0202 d causes a displayunit C0203 b to display a notification image based on thelight-receiving-unit conflict use information, via a display controlunit C0202 b (step SC0211).

FIG. 13 illustrates an example of a light receiving unit list stored bythe light reception control unit C0202 a. For each light receiving unitincluded in the terminal C0200, the light receiving unit list includes alight receiving unit ID_C1001 for identifying the light receiving unit,information C1002 indicating a side of the terminal C0200 on which thelight receiving unit is provided, and information C1003 indicating theuse status of the light receiving unit at this point in time. It shouldbe noted that the light receiving unit ID_C1001 illustrated in FIG. 13may be represented using one or more characters only, one or moresymbols only, a combination of one or more characters and one or morenumbers, a combination of one or more numbers and one or more symbols, acombination of one or more characters and one or more symbols, or acombination of one or more numbers, one or more characters, and one ormore symbols, rather than a number. In addition, the information C1002indicating the side of the terminal on which the light receiving unit isprovided in FIG. 13 may be represented using only one or more charactersindicating a side of a terminal, only one or more symbols indicating aside of a terminal, a combination of one or more characters and one ormore numbers indicating a side of a terminal, a combination of one ormore numbers and one or more symbols indicating a side of a terminal, acombination of one or more characters and one or more symbols indicatinga side of a terminal, or a combination of one or more numbers, one ormore characters, and one or more symbols indicating a side of aterminal, rather than characters. In addition, the information C1003indicating the use status of the light receiving unit illustrated inFIG. 13 may be represented using only one or more numbers indicating theuse status of a light receiving unit, only one or more symbolsindicating the use status of a light receiving unit, a combination ofone or more characters and one or more numbers indicating the use statusof a light receiving unit, a combination of one or more numbers and oneor more symbols indicating the use status of a light receiving unit, acombination of one or more characters and one or more symbols indicatingthe use status of a light receiving unit, or a combination of one ormore numbers, one or more characters, and one or more symbols indicatingthe use status of a light receiving unit, rather than characters.

FIG. 14 illustrates an example of a notification image displayed in stepSC0211 illustrated in FIG. 12. A notification image C1101 notifies auser that there are plural programs which are to use the light receivingunit C0203 a and furthermore, facilitates the user to select one of thelight receiving units C0203 a (on the front or back side) that is to beused by a focused program or to terminate the program (applicationprogram).

If the notification image C1101 is displayed in step SC0211 in FIG. 12,the user of the terminal C0200 selects which of the light receivingunits C0203 a is to be assigned to a program that causes conflict overuse of the light receiving unit C0203 a or selects to terminate aprogram that causes conflict over use of the light receiving unit C0203a. Thereafter, the selection result is input to an input unit C0203 c.The program control unit C0202 d determines whether the result of theuser selection has been input, via an input control unit C0202 c (stepSC0212). Here, the program control unit C0202 d receives the selectionresult if it is determined that the result has been input (YES in stepSC0212), and selects a light receiving unit to be used, according to theselection result (step SC0213).

FIG. 15 is a flowchart illustrating detailed processing operation ofstep SC0213 illustrated in FIG. 12. First, the program control unitC0202 d determines what the user has selected for a focused program(step SC0213 b). Here, if it is determined that a light receiving unitbeing used by another program is selected for the focused program, theprogram control unit C0202 d releases the light receiving unit C0203 aalready reserved by the other program (step SC0213 c), and clears, fromthe list of programs using a light receiving unit, information on thatother program registered in the program-using-light-receiving-unitinformation store unit C0202 e (step SC0213 d). Then, the programcontrol unit C0202 d adds information on the program (other program)from which the reserved light receiving unit is released, to thelight-receiving-unit conflict use information (step SC0213 e). Then, theprogram control unit C0202 d reserves the light receiving unit for thefocused program via the light reception control unit C0202 a (stepSC0213 f). Finally, the program control unit C0202 d registersinformation on the focused program into the list of programs using alight receiving unit in the program-using-light-receiving-unitinformation store unit C0202 e (step SC0213 g).

In step SC0213 b, if it is determined that the user has selected toterminate the program, the program control unit C0202 d terminates thefocused program (step SC0213 h). In addition, if it is determined instep SC0213 b that the user has selected the light receiving unitreserved by the focused program, the program control unit C0202 d skipssteps of the processing.

After executing the above processing, the program control unit C0202 ddeletes, from the light-receiving-unit conflict use information,information on a program which has selected the light receiving unitC0203 a or the terminated program (step SC0213 i).

Then, in step SC0214 illustrated in FIG. 12, the program control unitC0202 d determines whether the entire information included in thelight-receiving-unit conflict use information has been deleted (stepSC0214). Here, if the light-receiving-unit conflict use informationstill includes information on conflict over use of the light receivingunit C0203 a, the program control unit C0202 d executes, on theremaining information, processing from step SC0211 illustrated in FIG.12. In other words, the program control unit C0202 d changes a focusedprogram, and executes processing from step SC0211 on the changed focusedprogram. If the entire information included in the light-receiving-unitconflict use information is deleted, the program control unit C0202 dterminates the processing.

According to this variation as described above, in response to theoperation by a user, for example, the light receiving unit C0203 a onthe back side of the terminal C0200 is assigned to one of a visiblelight communication program C0201 a and a light-receiving-unit useprogram C0201 b which will cause conflict over use of the lightreceiving unit C0203 a on the front side of the terminal C0200, therebyeliminating the conflict. Then, if the light receiving unit C0203 a onthe back side is assigned to the visible light communication programC0201 a, the visible light communication program C0201 a causes acomputer to execute visible light communication according to the lightdetected by the light receiving unit C0203 a on the back side.

In this manner, the visible light communication program C0201 a and thelight-receiving-unit use program C0201 b are each allowed to use adifferent one of the light receiving unit C0203 a on the back side andthe light receiving unit C0203 a on the front side, and thus both theprograms are allowed to be activated simultaneously, without terminatingeither of the programs. Consequently, visible light communication andprocessing different from visible light communication can be executed inparallel, which achieves a reduction in a processing time.

Variation 3

An information communication method according to this variation is amethod for notifying, if the conflict over use of a light receiving unitoccurs, a user of occurrence of the conflict, and also terminating alight-receiving-unit use program activated later, after obtaining user'sconsent. It should be noted that a repeated description of the drawingsalready described in Embodiment 1 and Variations 1 and 2 thereof isavoided.

FIG. 16 is a flowchart illustrating processing operation by a terminalC0200 according to this variation.

Processing operation of the terminal C0200 according to this variationdiffers from the processing operation illustrated in FIG. 2 in thatinformation indicating whether light-receiving-unit conflict useinformation has been checked is received (step SC0311), as input from auser in response to a notification image displayed on a display unitC0203 b, rather than which program is permitted to use a light receivingunit.

In step SC0306 illustrated in FIG. 16, if conflict over use of the lightreceiving unit C0203 a is detected, the program control unit C0202 dcreates light-receiving-unit conflict use information based on a list ofprograms using a light receiving unit (step SC0309). Furthermore, theprogram control unit C0202 d causes the display unit C0203 b to displaya notification image according to the light-receiving-unit conflict useinformation, via the display control unit C0202 b.

FIG. 17 illustrates an example of a notification image displayed on thedisplay unit C0203 b according to this variation. A notification imageC1201 notifies a user that there are plural programs which are to usethe light receiving unit C0203 a, and furthermore, facilitates the userto check termination of a program activated later. The display of such anotification image notifies the user of the terminal C0200 whichincludes the light receiving unit C0203 a, that the conflict will occur.

The program control unit C0202 d terminates a program activated in stepSC0303 (program activated later) after receiving input indicating thatthe user has checked via the input unit C0203 c and the input controlunit C0202 c (step SC0312).

Also in this variation as described, as with Embodiment 1, if conflictover use of the light receiving unit C0203 a occurs, the user of thelight receiving unit C0203 a is notified of occurrence of the conflict,and a user operation permits a program activated earlier to use thelight receiving unit. Thus, the user can understand with ease the statusoccurring in the terminal C0200 that includes the light receiving unitC0203 a. Furthermore, the user understands the status, and then canmanipulate the terminal C0200. Thus, without performing extra operation,the user can solve with ease conflict over use of the light receivingunit C0203 a. As a result, improvement in a user interface can beachieved and burden on a user can be reduced.

Embodiment 2

An information communication method according to the present embodimentis a method for obtaining, based on an image obtained by, for example, areceiver achieved as a smartphone (multifunctional mobile phone)capturing an image of a subject that changes luminance, informationindicated by the luminance change. Here, the receiver obtainsinformation indicated by the luminance change of the subject if a markindicating a position of the subject is in a predetermined box in animage obtained by imaging.

FIGS. 18 and 19 illustrate examples of using a receiver according to thepresent embodiment.

A user of a receiver C1300 achieved as a smartphone finds a signboardC1320 that attract his/her interest. The signboard C1320 is an exampleof a data transmission light C0100, and transmits a signal by changingluminance. The user aims a camera (light receiving unit) of the receiverC1300 at the signboard C1320, and captures an image of the signboardC1320 as a subject. It should be noted that the receiver C1300 accordingto the present embodiment has the same configuration as the terminalC0200 according to Embodiment 1, and may perform visible lightcommunication using the visible light communication program C0201 a.

FIG. 20 illustrates an example of an image obtained by capturing animage.

The receiver C1300 displays a box C1302 and a mark C1301 indicating theposition of the signboard C1320 on the display of the receiver C1300, bycapturing an image of the signboard C1320. In other words, a capturedimage that includes the mark C1301 and the box C1302 is displayed on thedisplay of the receiver C1300. The mark C1301 is a subject image foridentifying the signboard C1320 which is a subject, and the box C1302 isdisplayed at a predetermined position of the display. Here, capturing animage is performed continuously and repeatedly at predetermined cycles.Thus, if a user moves the receiver C1300, the mark C1301 moves althoughthe position of the box C1302 on the display does not change. In otherwords, the receiver C1300 displays a captured image that includes themark C1301 for identifying the signboard C1320 and the box C1302 whilecontinuously updating the position of the mark C1301 on the capturedimage according to the positional relationship between the signboardC1320 and the image sensor of the camera established when the image iscaptured.

Then, the receiver C1300 according to the present embodiment receivessignals transmitted from the signboard C1320 by changing luminance anddemodulates the signals when the mark C1301 enters the box C1302,thereby obtaining information from the signboard C1320.

FIG. 21 illustrates a state when the mark C1301 enters the box C1302.

If it is determined that the mark C1301 has entered the box C1302, thereceiver C1300 changes the color of the box C1302 or displays the boxC1302 with a thicker line, in order to emphasize the box C1302, forexample. Then, the receiver C1300 receives a signal transmitted from thesignboard C1320 by changing luminance, which corresponds to the markC1301 that has entered the box C1302, and demodulates the signal,thereby obtaining information from the signboard C1320.

FIGS. 22 and 23 illustrate other examples of using a receiver accordingto the present embodiment.

A user of the receiver C1300 achieved as a smartphone finds thesignboard C1320 that attracts his/her interest. The signboard C1320transmits a signal by changing luminance. The user aims a camera (lightreceiving unit) of the receiver C1300 at the signboard C1320, andcaptures an image of the signboard C1320 as a subject. Here, a signboardC1321 is near the signboard C1320. The signboard C1321 is also anexample of a data transmission light C0100 and transmits a signal bychanging the luminance, as with the signboard C1320. Thus, the signboardC1321 may also be included in the imaging range of the camera of thereceiver C1300 aimed at the signboard C1320.

FIG. 24 illustrates another example of an image obtained by capturing animage.

The receiver C1300 captures an image of the signboard C1320, anddisplays, on the display of the receiver C1300, the box C1302 and themark C1301 indicating the position of the signboard C1320. The signboardC1321 is also included in the imaging range of the camera of thereceiver C1300, and thus a mark C1303 indicating the position of thesignboard C1321 is displayed on the display, together with the markC1301.

Here, the mark C1303 is a subject image for identifying the signboardC1321 which is a subject, and the box C1302 is displayed at apredetermined position on the display as with the description givenabove. In addition, capturing an image is performed continuously andrepeatedly at predetermined cycles. Thus, if the user moves the receiverC1300, the position of the box C1302 on the display does not change, butthe marks C1301 and C1303 move. Specifically, the receiver C1300displays a captured image which includes the box C1302 and the marksC1301 and C1303 for respectively identifying the signboards C1320 andC1321, according to a positional relationship between the image sensorof the camera and the signboards C1320 and C1321 established when theimages are captured, while continuously updating the positions of themarks C1301 and C1303 on the captured image.

When either of the marks C1301 and C1303 enters the box C1302, thereceiver C1300 according to the present embodiment receives anddemodulates a signal transmitted from a signboard by changing luminance,which corresponds to the mark in the box C1302, thereby obtaininginformation from the signboard.

FIG. 25 illustrates a state when the mark C1301 enters the box C1302.

If the receiver C1300 determines that the mark C1301 has entered the boxC1302, the receiver C1300 changes the color of the box C1302, ordisplays the box C1302 with a thicker line, to emphasize the box C1302,for example. Then, the receiver C1300 receives and demodulates only asignal transmitted from the signboard C1320 by changing luminance, whichcorresponds to the mark C1301 that has entered the box C1302 out of themarks C1301 and C1303, thereby obtaining only information from thesignboard C1320.

In this way, according to the information communication method accordingto the present embodiment, a visible light communication program whichruns in the background allows a user to easily select a subject (datatransmission light such as a signboard) from which data is to bereceived, for example. FIG. 26 is a flowchart illustrating processingoperation of the terminal C0200 achieved as the receiver C1300 accordingto the present embodiment.

First, the visible light communication program C0201 a performs imagingusing the light receiving unit C0203 a via the light reception controlunit C0202 a in a state where the shutter speed of the light receivingunit C0203 a is increased using the light reception control unit C0202 a(step SC0401). Next, the visible light communication program C0201 aobtains an image captured by the light receiving unit C0203 a via thelight reception control unit C0202 a (step SC0402). Imaging in stepSC0401 and obtaining a captured image in step SC0402 are repeatedlyperformed at fixed cycles.

Next, every time an image captured by the light receiving unit C0203 a(hereinafter, referred to as a captured image) is newly obtained, thevisible light communication program C0201 a locates an area in thecaptured image where the luminance exceeds a fixed threshold value andthe center coordinates thereof (step SC0403). The visible lightcommunication program C0201 a locates the center coordinates using amethod for locating the center coordinates as described in thefollowing, for example. According to the method for locating the centercoordinates, assuming that an X-axis is the width direction of acaptured image and a Y-axis is the height direction of a captured image,an x-coordinate located midway between an x-coordinate of a leftmostpixel and an x-coordinate of a rightmost pixel among pixels whereluminance exceeds a fixed threshold value is considered to be anx-coordinate of the center of an area where luminance exceeds a fixedthreshold value. Furthermore, among pixels where luminance exceeds afixed threshold value, a y coordinate located midway between a ycoordinate of an uppermost pixel and a y coordinate of a lowermost pixelis considered to be a y coordinate of the center of the area whereluminance exceeds a fixed threshold value.

Next, the visible light communication program C0201 a determines, viathe display control unit C0202 b, a position (hereinafter, referred toas a mark position) on the display unit C0203 b (display) correspondingto the center coordinates of the area where luminance exceeds the fixedthreshold value, and displays a mark at the mark position on the displayunit C0203 b (step SC0404). If a mark is already displayed on thedisplay unit C0203 b, the visible light communication program C0201 aonce deletes the mark that has been displayed until that time, andthereafter displays a mark at the newly calculated mark position. Forexample, the visible light communication program C0201 a associates thecenter of a captured image with the center of the display unit C0203 b,associates an upper end of the captured image with an upper end of thedisplay unit C0203 b, associates a lower end of the captured image witha lower end of the display unit C0203 b, associates a left end of thecaptured image with a left end of the display unit C0203 b, associates aright end of the captured image with a right end of the display unitC0203 b, and obtains a mark position from the ratio of the lengths ofthe sides of the captured image and the display unit C0203 b.Specifically, it is assumed that the width of the captured image is 480pixels, the height is 640 pixels, the upper left is the origin, thelower left coordinates are (0, 640), and the upper right coordinates are(480, 0). In addition, it is assumed that regarding the size of thedisplay unit C0203 b (display), the width is 960 pixels, the height is1280 pixels, the upper left is the origin, the lower left coordinatesare (0, 1280), and the upper right coordinates are (960, 0). Under thiscondition, if the above-described association is used, the visible lightcommunication program C0201 a determines the coordinates (200,400) asthe position (mark position) on the display unit C0203 b correspondingto the center coordinates (100, 200) of the above-described area on thecaptured image. The visible light communication program C0201 a displaysthe mark at the mark position every time a captured image is obtainedfrom the light receiving unit C0203 a via the light reception controlunit C0202 a.

In addition, the visible light communication program C0201 a displays,on the display unit C0203 b, the box C1302 illustrated in FIG. 20 (stepSC0405). The visible light communication program C0201 a compares themark position with the coordinates on the display unit C0203 b of thebox C1302 displayed on the display unit C0203 b, and determines whetherthe mark position is in the box C1302 (step SC0406). For example, thewidth of the display unit C0203 b is 960 pixels, and the height thereofis 1280 pixels. Then, in the coordinate system of the display unit C0203b, the upper left is the origin, the lower left coordinates are (0,1280), and the upper right coordinates are (960, 0). In addition, theupper left coordinates of the box C1302 being displayed are (450, 610),the lower left coordinates are (450, 670), the upper right coordinatesare (510, 610), and the lower right coordinates are (510, 670). At thistime, assuming that the mark position is (x, y), it is determined thatthe mark position is in the box C1302 if “450≦x≦510 and 610≦y≦670”.

Here, the visible light communication program C0201 a obtains a signaltransmitted by a data transmission light (such as a signboard)corresponding to an area where luminance exceeds the fixed thresholdvalue if it is determined that the mark position is in the box C1302(YES in step SC0406) (step SC0407).

For example, in the state illustrated in FIG. 20, the center coordinatesof the mark C1301 are outside the box C1302, and thus the visible lightcommunication program C0201 a does not receive a signal transmitted by adata transmission light corresponding to the mark C1301. On the otherhand, in the state illustrated in FIG. 21, the center coordinates of themark C1301 are in the box C1302, and thus the visible lightcommunication program C0201 a receives a signal transmitted by a datatransmission light corresponding to the mark C1301. In addition, asillustrated in FIG. 22, there are plural data transmission lights suchas signboards, and as illustrated in FIG. 23, the light receiving unitC0203 a simultaneously captures images of plural data transmissionlights. At this time, as illustrated in FIG. 24, if the centercoordinates of both of the marks C1301 and C1303 are outside the boxC1302, the visible light communication program C0201 a does not receivea signal from any of the data transmission lights. On the other hand, asillustrated in FIG. 25, if the center coordinates of the mark C1301 arepresent in the box C1302, the visible light communication program C0201a receives a signal from the data transmission light corresponding tothe mark C1301.

In the present embodiment, if images of plural data transmission lightsare captured by the light receiving unit C0203 a, marks are displayed atmark positions corresponding to respective data transmission lights.Marks are different for data transmission lights. the difference of themarks are represented by the difference in shape, the difference incolor, the difference in text, or the difference in illustration, forexample. Then, among the plural data transmission lights, a signaltransmitted from a data transmission light corresponding to a markposition determined to be in the box C1302 is received. If plural markpositions are present in the box C1302, the visible light communicationprogram C0201 a selects one of the positions, and receives a signaltransmitted from a data transmission light corresponding to the selectedmark position. For example, the visible light communication programC0201 a receives a signal transmitted from a data transmission lightcorresponding to the mark position which has entered the box C1302earlier. In addition, the visible light communication program C0201 areceives a signal transmitted from a data transmission light having alarge area where luminance exceeds the fixed threshold value.

It should be noted that although in the present embodiment, a datatransmission light from which a signal is obtained is selected bycomparing a mark position with the coordinates of a box displayed on thedisplay unit C0203 b, such a box may not be displayed. For example, amark position and the coordinates of a point on the display unit C0203 bdesignated by a user via the input unit C0203 c are compared, a datatransmission light corresponding to a mark position distant from themark position by a fixed range or less is selected as a datatransmission light from which a signal is to be obtained.

It should be noted that if an image captured by the light receiving unitC0203 a is displayed on the display unit C0203 b as it is, a markshowing an area where luminance exceeds the fixed threshold value maynot be displayed on the display unit C0203 b.

According to the present embodiment as described above, even if pluralsubjects which transmit signals by changing luminance are presentnearby, a signal only from a desired subject can be easily received bymerely capturing a subject image of the desired subject in a box.

Embodiment 3

Conventionally, reading a QR code (registered trademark) by using acamera has been used to automatically activate a data-usable application(application program) and make an access to an application installationsite in a terminal such as a smartphone. However, according to such aconventional method, it is necessary to take a close-up shot of a QRcode for reading, and thus reading from a distant location andsimultaneous reading by plural terminals are difficult.

According to the present embodiment, such a problem is addressed, andplural terminals are simultaneously allowed to automatically activate anapplication for which data can be used and automatically access anapplication installation site, without being brought close to a subjectwhose image is to be captured.

In other words, an information communication method according to thepresent embodiment includes: a light reception step of receiving visiblelight; a decoding step of extracting data from the received light; aninquiry step of inquiring of a server information regarding the obtaineddata and obtaining a result; an identification step of identifying adata-usable application or an installable application, using theobtained inquiry result; a display step of showing a data-usable orinstallable application to a user according to the identificationresult; and an execution step of activating or installing an applicationselected by a user who has received a notification. In this manner, aterminal which performs such an information communication method canobtain information on a data-usable or installable application, withoutbeing brought close to a subject to be imaged. In addition, pluralterminals which perform such an information communication method cansimultaneously obtain information on a data-usable or installableapplication.

The following describes the present embodiment in detail.

The present embodiment involves an information communication method forallowing a terminal to use data received from a data transmission light,displaying a list of applications installed in the terminal, andactivating an application selected by a user.

FIG. 27 illustrates a system configuration according to the presentembodiment.

A data transmission light C5100 transmits data using light in the samemanner as the data transmission light C0100 according to Embodiment 1. Aterminal C5200 includes: a light receiving unit C5201 which shoots lightemitted by the data transmission light C5100; a decoding unit C5205which extracts data from the light received by the light receiving unitC5201, as decoded data; a communication unit C5202 which transmits thedecoded data extracted by the decoding unit C5205 to a server C5300, andreceives, from the server C5300, a list of applications for which thedecoded data is usable (data-usable application list); a data-usableapplication search unit C5203 which detects an application installed inthe terminal C5200 from the data-usable application list received fromthe server C5300; an installed application DB_C5206 which storesinformation on an application installed in the terminal C5200; a displayunit C5204 which shows information to a user; an input unit C5208 whichreceives input from the user; and an application activation unit C5207which activates an application designated by the user via the input unitC5208. The light receiving unit C5201 has a function of changing theshutter speed used when receiving light. In addition, the terminal C5200may include a plurality of the light receiving units C5201.

The server C5300 includes: a communication unit C5301 which receivesdecoded data from the terminal C5200, and transmits a list ofapplications for which the decoded data is usable; a data-usableapplication detection unit C5302 which detects an application for whichdecoded data received from the terminal C5200 is usable; and adata-application association DB_C5303 which stores association betweendata transmitted by a data transmission light (decoded data) and anapplication for which the data is usable.

FIG. 28 is a flowchart illustrating processing operation by the terminalC5200 to receive visible light of visible light communication, andnotify a user of a data-usable application.

If a user aims the terminal C5200 at the data transmission light C5100as illustrated in FIG. 29, the terminal C5200 performs, using the lightreceiving unit C5201, processing for receiving visible light of visiblelight communication emitted from the data transmission light C5100 (stepSC5102). Next, the terminal C5200 determines whether visible light hasbeen received (step SC5103). Here, if it is determined that light hasbeen received (YES in step SC5103), the terminal C5200 performsprocessing for obtaining decoded data using the decoding unit C5205 fromthe light received by the light receiving unit C5201 (step SC5104).

FIG. 30 illustrates an example of decoded data to be obtained. Forexample, decoded data to be obtained is “1000.”

After step SC5104 illustrated in FIG. 28, the terminal C5200 determineswhether decoded data has been obtained (step SC5105). Here, if it isdetermined that decoded data has been obtained (YES in step SC5105), theterminal C5200 transmits, using the communication unit C5202, theobtained decoded data to the server C5300, and inquires whether there isany application for which the obtained decoded data is usable (stepSC5106). After that, the terminal C5200 receives from the server C5300using the communication unit C5202, a list of applications for which theobtained decoded data is usable (data-usable application list).

FIG. 31 illustrates an example of a data-usable application list that isreceived.

As illustrated in FIG. 31, this data-usable application list includesdecoded data C5501 and a data-usable application ID_C5502 foridentifying each of data-usable applications associated with the decodeddata.

In step SC5107 illustrated in FIG. 28, the terminal C5200 determinesbased on the data-usable application list received from the serverC5300, whether there are one or more applications for which the decodeddata obtained in step SC5104 is usable (step SC5107). Here, if it isdetermine that there are one or more such applications (YES in stepSC5107), the terminal C5200 compares, using the data-usable applicationsearch unit C5203, data-usable application IDs in the data-usableapplication list received from the server C5300 with application IDs inan installed application list stored in the installed applicationDB_C5206. Consequently, the terminal C5200 identifies an installedapplication from among one or more applications shown in the data-usableapplication list (data-usable application) (step SC5108).

FIG. 32 illustrates an example of information (installed applicationlist) stored in the installed application DB_C5206.

As illustrated in FIG. 32, the installed application list includes, foreach of the applications installed in the terminal C5200, an applicationID_C5601 for identifying the application and a name C5602 of theapplication (application name).

In step SC5109 illustrated in FIG. 28, the terminal C5200 determineswhether one or more data-usable applications are installed (stepSC5109). Then, if the terminal C5200 determines that one or moredata-usable applications are installed (YES in step SC5109), theterminal C5200 displays the one or more installed data-usableapplications on the display unit C5204 in the form of a list (stepSC5110).

FIG. 33 illustrates an example of a list of data-usable applicationsdisplayed on the display unit C5204.

The display unit C5204 shows names of the installed data-usableapplications, as illustrated in FIG. 33, for example. In this way, thescreen displayed on the display unit C5204 is an application selectionscreen for selecting an application program from among pluralapplication programs for which obtained decoded data is usable.

In step SC5111 illustrated in FIG. 28, the terminal C5200 determineswhether a user has selected, using the input unit C5208, an applicationthat uses the decoded data from among one or more data-usableapplications displayed on the display unit C5204 (step SC5111). Here, ifthe terminal C5200 determines that such an application has been selected(YES in step SC5111), the terminal C5200 activates the applicationselected by the user, using the application activation unit C5207 (stepSC5112).

It should be noted that if it is determined in step SC5107 that there isno application for which the decoded data obtained in step SC5104 isusable, or if it is determined in step SC5109 that there is no suchdata-usable application, the terminal C5200 shows, on display unit 5204,that there is no data-usable application (step SC5113).

FIG. 34 is a flowchart illustrating processing operation by the serverC5300 to search for an application for which decoded data is usable.

First, the server C5300 receives decoded data from the terminal C5200using the communication unit C5301 (step SC5302). Next, the server C5300searches information stored in the data-application association DB_C5303for an application (data-usable application) for which the decoded datais usable, using the data-usable application detection unit C5302 (stepSC5303).

FIG. 35 illustrates an example of information stored in thedata-application association DB_C5303.

This information includes, for each of applications, an applicationID_C5401 for identifying the application, and at least one piece ofdecoded data C5402 which is usable for the application.

Specifically, in step SC5303 illustrated in FIG. 34, the data-usableapplication detection unit C5302 of the server C5300 searchesinformation stored in the data-application association DB_C5303 fordecoded data received from the terminal C5200, and determines anapplication identified using an application ID associated with thedecoded data, as data-usable application.

Consequently, the server C5300 determines whether there is anydata-usable application (step SC5304). Here, if it is determined thatthere is a data-usable application (YES in step SC5304), the serverC5300 transmits to the terminal C5200 a data-usable application listshowing the data-usable application (step SC5305). On the other hand, ifit is determined that there is no data-usable application (NO in stepSC5304), the server C5300 transmits to the terminal C5200 informationfor notifying that there is no data-usable application (step SC5306).

According to the present embodiment as described above, the applicationselection screen illustrated in FIG. 33 is displayed, which is forselecting an application program from among plural application programsfor which obtained decoded data is usable. Then, if an applicationprogram is selected in response to a user operation on the applicationselection screen, the selected application program causes a computer toexecute processing using decoded data. In this manner, an appropriateapplication program can be caused to process data obtained by visiblelight communication.

Variation

This variation relates to an information communication method fordisplaying a list of applications not installed in a terminal amongapplications for which data received by a terminal from a datatransmission light is usable, and installing an application selected bya user.

FIG. 36 illustrates a system configuration according to this variation.

A data transmission light C7100 transmits data using light in the samemanner as the data transmission light C0100 according to Embodiment 1. Aterminal C7200 includes: a light receiving unit C7201 which shoots lightemitted by the data transmission light C7100; a decoding unit C7205which extracts data from the light received by the light receiving unitC7201 as decoded data; a communication unit C7202 which transmitsdecoded data extracted by the decoding unit C7205 to a server C7300, andreceives from the server C7300 a list of applications for which thedecoded data is usable (data-usable application list); an installableapplication detection unit C7203 which detects an applicationinstallable in the terminal C7200 from the data-usable application listreceived from the server C7300; an installed application DB_C7206storing information on an application installed in the terminal C7200; adisplay unit C7204 which shows information to a user; an input unitC7208 which receives input from the user; and an applicationinstallation unit C7207 which installs an application designated by theuser via the input unit C7208. The light receiving unit C7201 has afunction of changing the shutter speed used when receiving light. Inaddition, the terminal C7200 may include a plurality of the lightreceiving units C7201.

The server C7300 includes: a communication unit C7301 which receivesdecoded data from the terminal C7200, and transmits a list ofapplications for which decoded data is usable; a data-usable applicationdetection unit C7302 which detects an application for which decoded datareceived from the terminal C7200 is usable; and a data-applicationassociation DB_C7303 storing association between data transmitted by adata transmission light (decoded data) and an application for which thedata is usable.

FIG. 37 is a flowchart illustrating processing operation by the terminalC7200 to receive visible light of visible light communication, andnotify a user of a data-usable application.

If a user aims the terminal C7200 at the data transmission light C7100,the terminal C7200 performs processing for receiving visible light ofvisible light communication emitted from the data transmission lightC7100, using the light receiving unit C7201 (step SC7102). Next, theterminal C7200 determines whether visible light has been received (stepSC7103). Here, if it is determined that light has been received, (YES instep SC7103), the terminal C7200 performs processing for obtainingdecoded data illustrated in FIG. 30 from the light received by the lightreceiving unit C7201, using the decoding unit C7205 (step SC7104). Here,the terminal C7200 determines whether decoded data has been obtained(step SC7105).

If it is determined that decoded data has been obtained (YES in stepSC7105), the terminal C7200 transmits the obtained decoded data to theserver C7300 using the communication unit C7202, and inquires whetherthere is any application for which the obtained decoded data is usable(step SC7106). After that, the terminal C7200 receives a list ofdata-usable applications for which the obtained decoded data is usable(data-usable application list) from the server C7300 via thecommunication unit C7202.

FIG. 38 illustrates an example of a data-usable application list that isreceived.

As illustrated in FIG. 38, the data-usable application list includesdecoded data C7501, and for each of data-usable applications associatedwith the decoded data, an application ID_C7502 for identifying theapplication and an application name C7503 which is a name of theapplication.

In step SC7107 illustrated in FIG. 37, the terminal C5200 determines,based on the data-usable application list received from the serverC7300, whether there are one or more applications for which the decodeddata obtained in step SC7104 is usable (step SC7107). Here, if it isdetermined that there are one or more such applications (YES in stepSC7107), the terminal C7200 compares, using the installable applicationdetection unit C7203, data-usable application IDs on the data-usableapplication list received from the server C7300 with application IDs onthe installed application list stored in the installed applicationDB_C7206. Consequently, the terminal C7200 detects an application notinstalled in the terminal C7200 at that time and installable in theterminal C7200, from among the one or more applications shown by thedata-usable application list (data-usable application) (step SC7108).

FIG. 39 illustrates an example of information (installed applicationlist) stored in the installed application DB_C7206.

As illustrated in FIG. 39, the installed application list includes, foreach of applications installed in the terminal C7200, an applicationID_C7601 for identifying the application and a name C7602 of theapplication (application name).

In step SC7109 illustrated in FIG. 37, the terminal C7200 determineswhether there are one or more installable applications (step SC7109).For example, the terminal C7200 determines that there are one or moreinstallable applications (installable applications) if there is anapplication ID not included in the installed application list, among oneor more application IDs shown by the data-usable application list. Then,if it is determined that there are one or more installable applications(YES in step SC7109), the terminal C7200 displays the one or moreinstallable applications on the display unit C7204 in the form of a list(step SC7110).

FIG. 40 illustrates an example of a list of installable applicationsdisplayed on the display unit C7204.

For example, as illustrated in FIG. 40, the display unit C7204 displaysthe names of applications which can be installed.

In step SC7111 illustrated in FIG. 37, the terminal C7200 determineswhether the user has selected, using the input unit C7208, anapplication to be installed, from among one or more installableapplications displayed on the display unit C7204 (step SC7111). Here, ifthe terminal C5200 determines that the user has selected such anapplication (YES in step SC7111), the terminal C7200 installs theapplication selected by the user, using the application installationunit C7207 (step SC7112).

It should be noted that if it is determined in step SC7107 that there isno application for which the decoded data obtained in step SC7104 isusable, or if it is determined in step SC7109 that there is noinstallable application, the terminal C7200 shows that there is noinstallable application, on the display unit C7204 (step SC7113).

The server C7300 performs processing operation similar to processingoperation illustrated in FIG. 34, and generates a data-usableapplication list which includes the application names C7503 indicatingthe names of data-usable applications as illustrated in FIG. 38, andtransmits the list to the terminal C7200.

FIG. 41 illustrates an example of information stored in thedata-application association DB_C7303.

This information includes, for each of applications, an applicationID_C7401 for identifying the application, an application name C7402which is a name of the application, and at least one piece of decodeddata C7403 which is usable for the application. The data-usableapplication detection unit C7302 of the server C7300 detects one or moreapplications for which received decoded data is usable, based oninformation stored in the data-application association DB_C7303, andgenerates a data-usable application list that is a list of the one ormore detected applications, as illustrated in FIG. 38. In other words,the data-usable application detection unit C7302 searches informationstored in the data-application association DB_C7303 for the receiveddecoded data, and detects an application ID_C7401 and an applicationname C7402 associated with the decoded data. Then, the data-usableapplication detection unit C7302 generates a data-usable applicationlist which includes the detected application ID_C7401 and theapplication name C7402.

It should be noted that the application ID_C7502 illustrated in FIG. 38,the application ID_C7601 illustrated in FIG. 39, and the applicationID_C7401 illustrated in FIG. 41 may be package names of applications.

As described above, according to this variation, even if someapplication programs for which data obtained by visible lightcommunication is usable are not installed in a terminal, an applicationprogram selected from among the application programs in response to auser operation is installed. Thus, as with Embodiment 3 above, anappropriate application program can be caused to process data obtainedby visible light communication.

Embodiment 4

Light emission devices in a city such as streetlights, electric signs,signage, and car lights emit light as visible light transmissiondevices, thereby each transmitting a different visible light ID. In sucha case, there is a problem that it is not possible to obtain with ease avisible light ID only from a visible light transmission device fromwhich a user intends to obtain an ID. This problem arises since, inorder to receive light indicating a visible light ID, there is no methodfor obtaining, using a camera included in a mobile terminal such as asmartphone or a tablet terminal, a visible light ID of an only limitedvisible light transmission device from which a user intends to obtain anID from among plural visible light transmission devices captured by thecamera. In addition, also in the case where devices that can emit lightsuch as home electric appliances and lighting devices in a room areachieved as visible light transmission devices, it may be necessary toobtain a visible light ID only from a specific visible lighttransmission device, and thus a similar problem arises.

In view of this, in the present embodiment, association between agesture operation of a user toward a mobile terminal and a visible lighttransmission device from which a visible light ID is obtained ispreviously defined. In this manner, if a gesture operation by a usertoward a mobile terminal is detected when receiving a visible light ID,a visible light transmission device from which a visible light ID is tobe obtained can be limited to a visible light transmission deviceaccording to the intention of a user, by referring to the definedassociation.

In addition, a visible light transmission device from which a visiblelight ID is to be obtained can be automatically limited, and which oneof plural cameras included in the mobile terminal is to be used can bedetermined by detecting the orientation of the casing of a mobileterminal other than by an explicit operation of a user such as a gestureoperation, and thus implicit intention of the user can also beunderstood.

The following describes in detail an information communication methodaccording to the present embodiment, or in other words, a method forselecting a visible light ID in response to a gesture operation toward amobile terminal B0101.

FIG. 42 illustrates a system configuration according to the presentembodiment.

In the system illustrated in FIG. 42, a front camera B0106 or a backcamera B0107 included in the mobile terminal B0101 receive visible lightemitted by a visible light transmission device B0120, and avisible-light-ID receiving processing unit B0105 converts the visiblelight into a visible light ID. It should be noted that a visible lightID is an identifier for identifying detailed information associated withthe visible light ID.

On the other hand, in order to select a visible light ID, a gestureoperation by the user toward the mobile terminal B0101 is used. Sensingdata obtained by a sensing unit B0103 included in the mobile terminalB0101 is used in order to obtain a gesture operation. It should be notedthat the sensing unit B0103 includes a 9-axis sensor and others. Sensingdata obtained by the sensing unit B0103 is accumulated for a fixed timeperiod or at fixed times, and thereafter a gesture detection unit B0102detects (determines) a gesture operation based on the stored sensingdata. A detailed description is given of processing by the gesturedetection unit B0102 to detect a user gesture operation, using FIG. 46described below.

A received ID determination unit B0104 determines, based on a usergesture operation detected (determined) by the gesture detection unitB0102, whether detailed information associated with a visible light IDobtained by conversion by the visible-light-ID receiving processing unitB0105 is information that a user desires to know at present. If thedetermination shows that the detailed information associated with avisible light ID is necessary, the received ID determination unit B0104transmits an information obtaining request for obtaining informationassociated with a visible light ID to a communication unit B0108. Inthis manner, the communication unit B0108 transmits an informationobtaining request that includes a visible light ID to an ID associatedinformation conversion server B0111, via a public network B0130 andobtains visible light ID associated information (for example, URL) fordetailed information from the ID associated information conversionserver B0111.

On the other hand, if the detailed information associated with a visiblelight ID is unnecessary, or if the communication unit B0108 completesobtaining the visible light ID associated information from the IDassociated information conversion server B0111, the received IDdetermination unit B0104 requests a display unit B0109 for displayaccording to such cases.

A communication unit B0112 of the ID associated information conversionserver B0111 receives an information obtaining request that includes avisible light ID from the communication unit B0108 of the mobileterminal B0101. In this case, a conversion information determinationunit B0113 obtains visible light ID associated information associatedwith a visible light ID thereof from an ID associated information storeunit B0114. The conversion information determination unit B0113 deliversthe obtained visible light ID associated information to thecommunication unit B0112. As a result, the communication unit B0112transmits visible light ID associated information to the mobile terminalB0101 via the public network B0130.

FIG. 43 is a flowchart illustrating processing operation by the mobileterminal B0101, and in particular, a flowchart illustrating processingoperation for detecting a user gesture operation and selecting a visiblelight ID of received light.

Upon a start of processing for the mobile terminal B0101 to receive avisible light ID, the gesture detection unit B0102 monitors notificationof sensing data from the sensing unit B0103, thereby performingprocessing for detecting a gesture operation of the user toward themobile terminal B0101 (step SB0202). Next, the gesture detection unitB0102 determines whether a user gesture operation has been detected(step SB0203). If it is determined that such an operation has beendetected (YES in step SB0203), processing of step SB0204 is performed,whereas if it is determined that such an operation has not been detected(NO in step SB0203), processing of step SB0202 is performed repeatedly.A detailed description is given of operation of the gesture detectionunit B0102 using FIG. 46 described below.

Next, in step SB0204, the gesture detection unit B0102 notifies thereceived ID determination unit B0104 of the detected user gestureoperation (step SB0204). Next, the received ID determination unit B0104determines a light reception mode with reference to a gesture-modeassociation table stored thereby, which is illustrated in FIG. 49 laterdescribed. The visible-light-ID receiving processing unit B0105increases the shutter speed of a camera (front camera B0106 or backcamera B0107) which receives light indicating a visible light ID (stepSB0205).

The received ID determination unit B0104 determines whether notificationof a visible light ID has been received from the visible-light-IDreceiving processing unit B0105 (step SB0206). If it is determined thata notification has been received (YES in step SB0206), processing ofstep SB0207 is performed, whereas if it is determined that anotification has not been received (NO in step SB0206), processing ofstep SB0202 is performed repeatedly. In step SB0207, in order todetermine handling of a visible light ID according to a light receptionmode determined in step SB0205, the received ID determination unit B0104refers to an ID handling table stored thereby and illustrated in FIG. 50later described (step SB0207). In other words, the received IDdetermination unit B0104 determines, for each of notified visible lightIDs, whether the visible light ID is a visible light ID for whichdetailed information is required, or is a visible light ID for whichdetailed information is not required.

FIG. 44 is a flowchart illustrating processing operation of the mobileterminal B0101, and in particular a flowchart illustrating processingoperation for display.

After step SB0207 illustrated in FIG. 43, based on the result of theabove determination, the received ID determination unit B0104determines, for each of the notified visible light IDs, whether torequest the communication unit B0108 for an information obtainingrequest corresponding to the visible light ID (step SB0302). Here,processing of step SB0303 is performed if it is determined that such arequest is to be made (YES in step SB0302), whereas processing of stepSB0306 is performed if it is determined that such a request is not to bemade (NO in step SB0302). In step SB0303, the received ID determinationunit B0104 requests the communication unit B0108 to obtain, from the IDassociated information conversion server B0111, visible light IDassociated information corresponding to the visible light ID (stepSB0303). The communication unit B0108 makes an information obtainingrequest to the ID associated information conversion server B0111 via thepublic network B0130 (step SB0304). In other words, the communicationunit B0108 transmits, to the ID associated information conversion serverB0111, a command for requesting to transmit visible light ID associatedinformation associated with a visible light ID to the communication unitB0108 (information obtaining request that includes a visible light ID).

Next, if notification information in response to the informationobtaining request is transmitted from the ID associated informationconversion server B0111, the communication unit B0108 receivesnotification information from the ID associated information conversionserver B0111 via the public network B0130 (step SB0311).

Then, after step SB0311 or if it is determined in step SB0302 that arequest is not to be made (NO in step SB0302), the received IDdetermination unit B0104 determines, for each of notified visible lightIDs, whether detailed information associated with the visible light IDis to be displayed on the display unit B0109 (step SB0306). For example,if a visible light ID is determined to be a visible light ID for whichdetailed information is necessary, the received ID determination unitB0104 determines that detailed information associated with the visiblelight ID is to be displayed on the display unit B0109. In contrast, thereceived ID determination unit B0104 determines that detailedinformation associated with the visible light ID is not to be displayedon the display unit B0109 when the visible light ID is determined to bea visible light ID for which detailed information is not necessary. Inaddition, the received ID determination unit B0104 determines thatdetailed information associated with the visible light ID is to bedisplayed on the display unit B010 if notification informationcorresponding to the visible light ID is received in step SB03111.

Here, processing of step SB0307 is performed if it is determined thatdetailed information corresponding to the visible light ID is to bedisplayed on the display unit B0109 (YES in step SB0306), whereas if itis determined that such information is not to be displayed (NO in stepSB0306), the mobile terminal B0101 terminates processing of displayingdetailed information associated with the visible light ID.

In step SB0307, the received ID determination unit B0104 deliversnotification information corresponding to a visible light ID to thedisplay unit B0109 (step SB0307). Then, the display unit B0109 displaysdetailed information (visible light ID associated information) which canbe displayed in correspondence with the visible light ID, based on thereceived notification information, and terminates the processing.Specifically, if notification information includes visible light IDassociated information and the visible light ID associated informationis a uniform resource locator (URL), the mobile terminal B0101 obtainsdetailed information from the location indicated by the URL, anddisplays the detailed information on the display unit B0109.

FIG. 45 is a flowchart illustrating processing operation of the IDassociated information conversion server B0111 performed when receivingan information obtaining request from the mobile terminal B0101.

The communication unit B0112 of the ID associated information conversionserver B0111 determines whether an information obtaining request hasbeen received from the mobile terminal B0101 (step SB0402). Here, if itis determined that the request has been received (YES in step SB0402),processing of step SB0403 is performed, whereas if it is determined thatthe request has not been received (NO in step SB0402), processing ofstep SB0402 is performed repeatedly.

In step SB0403, the communication unit B0112 notifies the conversioninformation determination unit B0113 of the visible light ID included inthe received information obtaining request (step SB0403). Next, theconversion information determination unit B0113 inquires of the IDassociated information store unit B0114 about the visible light IDassociated information associated with the notified visible light ID(step SB0404). Here, the ID associated information store unit B0114determines whether the notified visible light ID is included in an IDassociated information table stored therein (step SB0405). Here, if itis determined that a visible light ID is included in the table (YES instep SB0405), processing of step SB0406 is performed, whereas if it isdetermined that a visible light ID is not included in the table (NO instep SB0405), processing of step SB0409 is performed.

In step SB0406, the ID associated information store unit B0114 notifiesthe conversion information determination unit B0113 of visible light IDassociated information associated with the visible light ID, obtainedfrom the ID associated information table illustrated in FIG. 51 laterdescribed (step SB0406). Next, the conversion information determinationunit B0113 generates notification information which includes the visiblelight ID associated information, and delivers the information to thecommunication unit B0112 (step SB0407).

In step SB0409, the ID associated information store unit B0114 notifiesthe conversion information determination unit B0113 that the IDassociated information table illustrated FIG. 51 described later doesnot include the visible light ID (step SB0409). Next, the conversioninformation determination unit B0113 generates notification informationindicating that visible light ID associated information of the ID is notincluded, and delivers the notification information to the communicationunit B0112 (step SB0410).

After step SB0407 or SB0410, the communication unit B0112 returns thenotification information received from the conversion informationdetermination unit B0113 to the mobile terminal B0101 that has made aninformation obtaining request (step SB0408).

FIG. 46 is a flowchart illustrating processing operation by the mobileterminal B0101 to detect a user gesture operation. The gesture operationdetected by this processing operation is a gesture made when a usermanipulates the mobile terminal B0101, and used in order to determine avisible light ID obtained through light reception by the mobile terminalB0101.

Upon the start of the gesture detection unit B0102 detecting a gestureoperation of the user, the gesture detection unit B0102 initializes thenumber of times sensing data is received to 0 (step SB0502). Then, thegesture detection unit B0102 enters a state of awaiting sensing datafrom the sensing unit B0103 (data reception waiting state) (stepSB0503). Next, the gesture detection unit B0102 determines whethersensing data has been received from the sensing unit B0103 (stepSB0504). Here, if it is determined that such data has been received (YESin step SB0504), processing of step SB0505 is performed, whereas if itis determined that such data has not been received (NO in step SB0504),processing of step SB0503 is performed repeatedly.

In step SB0505, the gesture detection unit B0102 temporarily holdssensing data, and also adds 1 to the number of times sensing data isreceived (step SB0505). Then, the gesture detection unit B0102determines whether the number of times sensing data is received is apredetermined number of times or more (step SB0506). It should be notedthat in step SB0502, the gesture detection unit B0102 may initialize asensing-data reception elapsed time that is a time elapsed since thestart of awaiting sensing data to 0, instead of initializing the numberof times sensing data is received to 0. In this case, in step S50506,the gesture detection unit B0102 determines whether a sensing-datareception elapsed time is longer than or equal to a predetermined time.If it is determined that the number of times sensing data is received isa predetermined number of times or more, or if it is determined that asensing data reception elapsed time is longer than or equal to apredetermined time (YES in step SB0506), processing of step SB0507 isperformed, whereas if it is determined that the number of times sensingdata is received is less than a prescribed number of times, or if it isdetermined that a sensing-data reception elapsed time is less than apredetermined time (NO in step SB0506), processing of step SB0503 isperformed repeatedly.

Next, the gesture detection unit B0102 detects a current gestureoperation using at least one piece of sensing data temporarily stored(step SB0507). At this time, the gesture detection unit B0102 storesthat detected gesture operation as a current gesture operation. Then,the gesture detection unit B0102 deletes at least one piece of sensingdata temporarily stored (step SB0508).

Here, the gesture detection unit B0102 determines whether a signalindicating a request for stopping the detection of a gesture operation(detection stop request signal) has been received from the received IDdetermination unit B0104 (step SB0509). Here, if it is determined thatsuch a signal has been received (YES in step SB0509), the gesturedetection unit B0102 terminates processing operation for detecting agesture operation. On the other hand, if it is determined that such asignal has not been received (NO in step SB0509), the gesture detectionunit B0102 repeatedly executes processing from step SB0502. It should benoted that regarding processing operation for detecting a gestureoperation illustrated in FIG. 46, the elapse of a fixed time period orinterrupt processing performed by the sensing unit B0103 triggers thestart of processing from step SB0502, again.

FIG. 47 is a diagram for describing operation of selecting a visiblelight ID by the mobile terminal B0101 in response to a gestureoperation.

For example, streetlights B0605 to B0608 and taxis B0601 to B0603 eachhave the visible light transmission device B0120 and emit visible light,thereby transmitting a visible light ID. Here, a terminal which includesa general conventional camera obtains pieces of detailed informationcorresponding to visible light IDs of all the visible light transmissiondevices that are in a direction in which camera shooting is performed.However, according to the present embodiment, the mobile terminal B0101limits detailed information to be obtained only to pieces of detailedinformation of the taxis B0601 to B0603 by using a combination ofintention of a user that he/she desires to obtain only detailedinformation of the taxis B0601 to B0603 and a gesture operation of auser moving the mobile terminal B0101 upward (in the arrow direction inFIG. 47), for example.

This can limit obtaining detailed information unnecessary for a user. Inother words, detailed information (visible light ID associatedinformation) displayed on the display unit B0109 can be selected, whichachieves effects such as an improvement in the viewability of a userinterface and limitation of obtaining visible light ID associatedinformation via a network. It should be noted that cameras (the frontcamera B0106 and the back camera B0107) perform processing of changingthe shutter speed to a high speed when receiving a visible light ID.

FIG. 48 illustrates an example of processing operation of selectingdetailed information displayed in response to a gesture operation.

The mobile terminal B0101 obtains detailed information of only the taxisB0601 to B0603 if the mobile terminal B0101 detects a gesture operationof moving the mobile terminal B0101 upward, as described above. Then,the mobile terminal B0101 displays a captured image shot by a camera(the front camera B0106, the back camera B0107). At this time, themobile terminal B0101 superimposes balloon images(information-presentable notification images) B0702 to B0704 showingthat detailed information is obtained, on the captured image to bedisplayed, in a state where the balloon images are respectivelyassociated with the images of the taxis B0601 to B0603. Furthermore, themobile terminal B0101 displays detailed information of only one of thetaxis using a balloon image of the taxi, among the detailed informationof the taxis B0601 to B0603. For example, detailed information of thetaxi B0601, that is, “Now, 10% discount” is displayed in the balloonimage B0702. A balloon image that includes such detailed information isan image generated according to a visible light ID from a taxi which isa subject, and an information notification image showing detailsregarding the taxi. In addition, the above-describedinformation-presentable notification image (balloon image which does notinclude detailed information) is an image for notifying that detailedinformation can be shown.

Here, if a user shakes the mobile terminal B0101 in the upper rightdirection (arrow direction in FIG. 48), the gesture detection unit B0102of the mobile terminal B0101 detects a gesture operation of shaking inthe upper right direction. As a result, the mobile terminal B0101 hidesdetailed information displayed in the balloon image B0702. Furthermore,at this time, the mobile terminal B0101 displays detailed information ofthe taxi B0602 associated with the balloon image B0703, that is, “Now,long distance discount” in the balloon image B0703 closest to theballoon image B0702 in the upper right direction on the captured image.

As described above, in the present embodiment, a user can easily changedetailed information to be displayed by a gesture operation of shakingthe mobile terminal B0101.

FIG. 49 illustrates an example of a gesture-mode association tablestored by the received ID determination unit B0104.

A gesture-mode association table shows, for each gesture operation, agesture operation B0801 and a light reception mode B0802 associated withthe gesture operation. For example, a light reception mode “taxi” fordisplaying detailed information of a taxi is associated with a gestureoperation “hold up” that is an operation of holding up a mobile terminalin front of a user (or operation of raising a mobile terminal upwards).In addition, a light reception mode “cancel” for canceling a lightreception mode currently set is associated with a shake which is anoperation of laterally shaking a mobile terminal.

FIG. 50 illustrates an example of an ID handling table stored by thereceived ID determination unit B0104.

This ID handling table includes, for each light reception mode, a lightreception mode B0901, a first group ID_B0902 for identifying the lightreception mode, a name B0903 of a visible light transmission devicewhich transmits a visible light ID that is a candidate in the lightreception mode, a second group ID_B0904 for identifying the visiblelight transmission device, information 80905 indicating whether detailedinformation based on the visible light transmission device is to bedisplayed on the display unit B0109, and information B0906 indicatingwhether detailed information of the light reception mode is to beobtained from the ID associated information conversion server B0111. Forexample, the first group ID “TAXIAA1” and the names of visible lighttransmission devices, namely, a “roof lamp” and a “headlight” areassociated with a light reception mode (taxi) for displaying detailedinformation of a taxi. In addition, the name “roof lamp” of a visiblelight transmission device is associated with the second group ID“XX001”, “necessary” indicating that detailed information based on theroof lamp is to be displayed on the display unit B0109, and “necessary”indicating that detailed information thereof is to be obtained from theID associated information conversion server B0111.

FIG. 51 illustrates an example of an ID associated information tablestored by the ID associated information store unit B0114.

This ID associated information table includes, for each visible lightID, a visible light ID_B1001 thereof, and visible light ID associatedinformation B1002 associated with the visible light ID. As illustratedin the ID associated information table, a visible light ID includesthree alphanumeric strings connected with hyphens (-). The firstalphanumeric string of the alphanumeric strings corresponds to the firstgroup ID_B0902 illustrated in FIG. 50, the middle alphanumeric stringcorresponds to a second group ID_B0904 illustrated in FIG. 50, and thelast alphanumeric string is an ID for identifying visible light IDassociated information. Visible light ID associated information isindicated by URL or the like, for example. If the visible light IDassociated information included in notification information is URL, themobile terminal B0101 obtains detailed information from the locationindicated by the URL. In addition, among the visible light IDs B1001included in the ID associated information table illustrated in FIG. 51,“TAXIAA1-XX001-0001”, “TAXIAA1-XX001-0002”, “TAXIAA1-XX001-0003”, and“TAXIAA1-XX001-0004” are visible light IDs transmitted by visible lightfrom roof lamps of taxis. In addition, among the visible light IDs B1001included in the ID associated information table illustrated in FIG. 51,“TAXIAA1-HR001-0001” and “TAXIAA1-HR001-0002” are visible light IDstransmitted by visible light from headlights of taxis.

It should be noted that although a light reception mode is determined bya user gesture operation in the above embodiment, other factors maydetermine a light reception mode. For example, the orientation of themobile terminal B0101 may determine a light reception mode.

FIG. 52 is a diagram for describing processing of determining a lightreception mode by the orientation of the mobile terminal B0101.

The sensing unit B0103 of the mobile terminal B0101 detects theorientation of a casing of the mobile terminal B0101. The mobileterminal B0101 determines, based on the detection result, one of thefront camera B0106 and the back camera B0107 to be a camera whichreceives visible light B1101 from the visible light transmission deviceB0120. In this manner, only one of the front camera B0106 and the backcamera B0107 receives the visible light B1101. Consequently, from amongvisible light IDs obtained by the front camera B0106 and the back cameraB0107, a visible light ID corresponding to detailed informationnecessary for a user can be easily selected according to the orientationof the mobile terminal B0101. Furthermore, induction of the fraudulentaction by reflected light and the like can be reduced, and powerconsumption can also be reduced.

For example, as illustrated in (a) of FIG. 52, the mobile terminal B0101activates only the front camera B0106 if a state is detected where thecasing is tilted at 45 degrees or more (for example, 60 degrees)relative to the vertical direction with the front side facing upwards,as the orientation of the casing of the mobile terminal B0101. On theother hand, as illustrated in (b) of FIG. 52, the mobile terminal B0101activates only the back camera B0107 if a state is detected where thecasing is tilted at 45 degrees or less (for example, 10 degrees)relative to the vertical direction with the front side facing a user, asthe orientation of the casing of the mobile terminal B0101.

It should be noted that in the present embodiment, a description isgiven of the case where only detailed information corresponding to avisible light ID transmitted by a taxi emitting visible light isobtained. However, also in addition to such detailed information,detailed information corresponding to only a visible light IDtransmitted by store signboards and signage in a town emitting visiblelight may be obtained, and detailed information corresponding to only avisible light ID transmitted by home electric appliances (for example, alight emitting diode (LED) lamp and a video on a television) and othersin a room emitting visible light may be obtained.

As described above, in the present embodiment, among signals (visiblelight IDs) transmitted by plural subjects (such as taxis) and received,an information notification image which is an image generated accordingto a signal from a subject, and an image indicating informationregarding the subject (a balloon image which includes detailedinformation) is superimposed on a captured image in the state where theinformation notification image is associated with an image of thesubject. Furthermore, when it is determined that a terminal device(mobile terminal) has been shaken, another information notificationimage that is an image generated according to a signal from anothersubject in the direction in which the terminal device is shaken, andshowing information regarding that other subject is superimposed on acaptured image in a state where the information notification image isassociated with an image of the other subject. Consequently, forexample, an information notification image having a description ofdetailed information regarding a subject in a balloon is displayed in astate where the information notification image is associated with animage of the subject, and thus a user can appropriately understandinformation regarding the subject. In addition, a subject for which aninformation notification image is to be displayed can be easily changedby shaking a terminal device. Furthermore, the subject is changed to asubject that is in the direction in which the terminal device is shaken,thus allowing a user to intuitively make such a change and achieving afurther improvement in a user interface.

Furthermore, in the present embodiment, for each of the images ofremaining subjects among images of plural subjects, excluding an imageof a subject associated with an information notification image, ainformation-presentable notification image for notifying thatinformation (detailed information) regarding the remaining subjects canbe presented are superimposed on a captured image. Thus, for example, aninformation-presentable notification image shaped like a small balloonhaving no detailed information is displayed for an image of a subjectfor which an information notification image is not displayed, and thus auser can determine that information regarding the subject (detailedinformation) can be displayed as an information notification image byshaking a terminal device.

FIG. 53A is a flowchart of an information communication method accordingto an aspect of the present disclosure.

An information communication method according to an aspect of thepresent disclosure is an information communication method for performingvisible light communication, and includes steps S11, S12, S13, and S14.

Specifically, this information communication method includes:determining whether conflict over use of a light receiving unit whichdetects light will occur between a first program for performing visiblelight communication using the light receiving unit and a second programfor performing processing different from the visible lightcommunication, using the light receiving unit (S11); notifying a user ofthe light receiving unit that the conflict will occur if it isdetermined that the conflict will occur (S12); permitting one of thefirst program and the second program to use the light receiving unit inresponse to an operation by the user (S13); and causing, by the firstprogram, a computer to execute visible light communication according tolight detected by the light receiving unit if the first program ispermitted to use the light receiving unit (S14).

FIG. 53B is a block diagram illustrating an information communicationdevice according to an aspect of the present disclosure.

An information communication device 10 according to an aspect of thepresent disclosure is an information communication device which performsvisible light communication, and includes constituent elements 11, 12,13, and 14.

Specifically, the information communication device 10 includes: adetermination unit 11 configured to determine whether conflict over useof a light receiving unit configured to detect light will occur betweena first program for performing visible light communication using thelight receiving unit and a second program for performing processingdifferent from the visible light communication, using the lightreceiving unit; a notification unit 12 configured to notify a user ofthe light receiving unit that the conflict will occur if it isdetermined that the conflict will occur; a conflict settlement unit 13configured to permit one of the first program and the second program touse the light receiving unit in response to an operation by the user;and a visible light communication unit 14 configured to cause, accordingto the first program, a computer to execute visible light communicationaccording to light detected by the light receiving unit if the firstprogram is permitted to use the light receiving unit.

According to the information communication method and the informationcommunication device 10 illustrated in FIGS. 53A and 53B as describedabove, if conflict over use of a light receiving unit will occur, a userof the light receiving unit is notified of the conflict, and a useroperation permits one of programs to use the light receiving unit. Thus,the user can understand with ease the status occurring in the terminalthat includes the light receiving unit. Furthermore, the userunderstands the status and then can manipulate the terminal. Thus,conflict over use of a light receiving unit can be easily solved withoutperforming an extra operation. As a result, a user interface can beimproved and the burden on a user can be reduced.

It should be noted that in the above embodiments and variations, each ofthe constituent elements may be constituted by dedicated hardware or maybe obtained by executing a software program suitable for the constituentelement. Each constituent element may be obtained by a program executionunit such as a CPU or a processor reading and executing a softwareprogram recorded on a recording medium such as a hard disk orsemiconductor memory. For example, a program causes a computer toexecute the information communication method illustrated by theflowchart in FIG. 53A.

Embodiment 5

The following describes Embodiment 5.

(Observation of Luminance of Light Emitting Unit)

The following proposes an imaging method in which, when capturing oneimage, all imaging elements are not exposed simultaneously but the timesof starting and ending the exposure differ between the imaging elements.FIG. 54 illustrates an example of imaging where imaging elementsarranged in a line are exposed simultaneously, with the exposure starttime being shifted in order of lines. Here, the simultaneously exposedimaging elements are referred to as “exposure line”, and the line ofpixels in the image corresponding to the imaging elements is referred toas “bright line”.

In the case of capturing a blinking light source shown on the entireimaging elements using this imaging method, bright lines (lines ofbrightness in pixel value) along exposure lines appear in the capturedimage as illustrated in FIG. 55. By recognizing this bright linepattern, the luminance change of the light source at a speed higher thanthe imaging frame rate can be estimated. Hence, transmitting a signal asthe luminance change of the light source enables communication at aspeed not less than the imaging frame rate. In the case where the lightsource takes two luminance values to express a signal, the lowerluminance value is referred to as “low” (LO), and the higher luminancevalue is referred to as “high” (HI). The low may be a state in which thelight source emits no light, or a state in which the light source emitsweaker light than in the high.

By this method, information transmission is performed at a speed higherthan the imaging frame rate.

In the case where the number of exposure lines whose exposure times donot overlap each other is 20 in one captured image and the imaging framerate is 30 fps, it is possible to recognize a luminance change in aperiod of 1.67 millisecond. In the case where the number of exposurelines whose exposure times do not overlap each other is 1000, it ispossible to recognize a luminance change in a period of 1/30000 second(about 33 microseconds). Note that the exposure time is set to less than10 milliseconds, for example.

FIG. 55 illustrates a situation where, after the exposure of oneexposure line ends, the exposure of the next exposure line starts.

In this situation, when transmitting information based on whether or noteach exposure line receives at least a predetermined amount of light,information transmission at a speed of fl bits per second at the maximumcan be realized where f is the number of frames per second (frame rate)and l is the number of exposure lines constituting one image.

Note that faster communication is possible in the case of performingtime-difference exposure not on a line basis but on a pixel basis.

In such a case, when transmitting information based on whether or noteach pixel receives at least a predetermined amount of light, thetransmission speed is flm bits per second at the maximum, where m is thenumber of pixels per exposure line.

If the exposure state of each exposure line caused by the light emissionof the light emitting unit is recognizable in a plurality of levels asillustrated in FIG. 56, more information can be transmitted bycontrolling the light emission time of the light emitting unit in ashorter unit of time than the exposure time of each exposure line.

In the case where the exposure state is recognizable in Elv levels,information can be transmitted at a speed of flElv bits per second atthe maximum.

Moreover, a fundamental period of transmission can be recognized bycausing the light emitting unit to emit light with a timing slightlydifferent from the timing of exposure of each exposure line.

FIG. 57A illustrates a situation where, before the exposure of oneexposure line ends, the exposure of the next exposure line starts. Thatis, the exposure times of adjacent exposure lines partially overlap eachother. This structure has the feature (1): the number of samples in apredetermined time can be increased as compared with the case where,after the exposure of one exposure line ends, the exposure of the nextexposure line starts. The increase of the number of samples in thepredetermined time leads to more appropriate detection of the lightsignal emitted from the light transmitter which is the subject. In otherwords, the error rate when detecting the light signal can be reduced.The structure also has the feature (2): the exposure time of eachexposure line can be increased as compared with the case where, afterthe exposure of one exposure line ends, the exposure of the nextexposure line starts. Accordingly, even in the case where the subject isdark, a brighter image can be obtained, i.e. the S/N ratio can beimproved. Here, the structure in which the exposure times of adjacentexposure lines partially overlap each other does not need to be appliedto all exposure lines, and part of the exposure lines may not have thestructure of partially overlapping in exposure time. By keeping part ofthe exposure lines from partially overlapping in exposure time, theoccurrence of an intermediate color caused by exposure time overlap issuppressed on the imaging screen, as a result of which bright lines canbe detected more appropriately.

In this situation, the exposure time is calculated from the brightnessof each exposure line, to recognize the light emission state of thelight emitting unit.

Note that, in the case of determining the brightness of each exposureline in a binary fashion of whether or not the luminance is greater thanor equal to a threshold, it is necessary for the light emitting unit tocontinue the state of emitting no light for at least the exposure timeof each line, to enable the no light emission state to be recognized.

FIG. 57B illustrates the influence of the difference in exposure time inthe case where the exposure start time of each exposure line is thesame. In 7500 a, the exposure end time of one exposure line and theexposure start time of the next exposure line are the same. In 7500 b,the exposure time is longer than that in 7500 a. The structure in whichthe exposure times of adjacent exposure lines partially overlap eachother as in 7500 b allows a longer exposure time to be used. That is,more light enters the imaging element, so that a brighter image can beobtained. In addition, since the imaging sensitivity for capturing animage of the same brightness can be reduced, an image with less noisecan be obtained. Communication errors are prevented in this way.

FIG. 57C illustrates the influence of the difference in exposure starttime of each exposure line in the case where the exposure time is thesame. In 7501 a, the exposure end time of one exposure line and theexposure start time of the next exposure line are the same. In 7501 b,the exposure of one exposure line ends after the exposure of the nextexposure line starts. The structure in which the exposure times ofadjacent exposure lines partially overlap each other as in 7501 b allowsmore lines to be exposed per unit time. This increases the resolution,so that more information can be obtained. Since the sample interval(i.e. the difference in exposure start time) is shorter, the luminancechange of the light source can be estimated more accurately,contributing to a lower error rate. Moreover, the luminance change ofthe light source in a shorter time can be recognized. By exposure timeoverlap, light source blinking shorter than the exposure time can berecognized using the difference of the amount of exposure betweenadjacent exposure lines.

As described with reference to FIGS. 57B and 57C, in the structure inwhich each exposure line is sequentially exposed so that the exposuretimes of adjacent exposure lines partially overlap each other, thecommunication speed can be dramatically improved by using, for signaltransmission, the bright line pattern generated by setting the exposuretime shorter than in the normal imaging mode. Setting the exposure timein visible light communication to less than or equal to 1/480 secondenables an appropriate bright line pattern to be generated. Here, it isnecessary to set (exposure time)<⅛×f, where f is the frame frequency.Blanking during imaging is half of one frame at the maximum. That is,the blanking time is less than or equal to half of the imaging time. Theactual imaging time is therefore ½f at the shortest. Besides, since4-value information needs to be received within the time of ½f, it isnecessary to at least set the exposure time to less than 1/(2f×4). Giventhat the normal frame rate is less than or equal to 60 frames persecond, by setting the exposure time to less than or equal to 1/480second, an appropriate bright line pattern is generated in the imagedata and thus fast signal transmission is achieved.

FIG. 57D illustrates the advantage of using a short exposure time in thecase where each exposure line does not overlap in exposure time. In thecase where the exposure time is long, even when the light source changesin luminance in a binary fashion as in 7502 a, an intermediate-colorpart tends to appear in the captured image as in 7502 e, making itdifficult to recognize the luminance change of the light source. Byproviding a predetermined non-exposure blank time (predetermined waittime) t_(D2) from when the exposure of one exposure line ends to whenthe exposure of the next exposure line starts as in 7502 d, however, theluminance change of the light source can be recognized more easily. Thatis, a more appropriate bright line pattern can be detected as in 7502 f.The provision of the predetermined non-exposure blank time is possibleby setting a shorter exposure time t_(E) than the time difference t_(D)between the exposure start times of the exposure lines, as in 7502 d. Inthe case where the exposure times of adjacent exposure lines partiallyoverlap each other in the normal imaging mode, the exposure time isshortened from the normal imaging mode so as to provide thepredetermined non-exposure blank time. In the case where the exposureend time of one exposure line and the exposure start time of the nextexposure line are the same in the normal imaging mode, too, the exposuretime is shortened so as to provide the predetermined non-exposure time.Alternatively, the predetermined non-exposure blank time (predeterminedwait time) t_(D2) from when the exposure of one exposure line ends towhen the exposure of the next exposure line starts may be provided byincreasing the interval t_(D) between the exposure start times of theexposure lines, as in 7502 g. This structure allows a longer exposuretime to be used, so that a brighter image can be captured. Moreover, areduction in noise contributes to higher error tolerance. Meanwhile,this structure is disadvantageous in that the number of samples is smallas in 7502 h, because fewer exposure lines can be exposed in apredetermined time. Accordingly, it is desirable to use these structuresdepending on circumstances. For example, the estimation error of theluminance change of the light source can be reduced by using the formerstructure in the case where the imaging object is bright and using thelatter structure in the case where the imaging object is dark.

Here, the structure in which the exposure times of adjacent exposurelines partially overlap each other does not need to be applied to allexposure lines, and part of the exposure lines may not have thestructure of partially overlapping in exposure time. Moreover, thestructure in which the predetermined non-exposure blank time(predetermined wait time) is provided from when the exposure of oneexposure line ends to when the exposure of the next exposure line startsdoes not need to be applied to all exposure lines, and part of theexposure lines may have the structure of partially overlapping inexposure time. This makes it possible to take advantage of each of thestructures. Furthermore, the same reading method or circuit may be usedto read a signal in the normal imaging mode in which imaging isperformed at the normal frame rate (30 fps, 60 fps) and the visiblelight communication mode in which imaging is performed with the exposuretime less than or equal to 1/480 second for visible light communication.The use of the same reading method or circuit to read a signaleliminates the need to employ separate circuits for the normal imagingmode and the visible light communication mode. The circuit size can bereduced in this way.

FIG. 57E illustrates the relation between the minimum change time t_(S)of light source luminance, the exposure time t_(E), the time differencet_(D) between the exposure start times of the exposure lines, and thecaptured image. In the case where t_(E)+t_(D)<t_(S), imaging is alwaysperformed in a state where the light source does not change from thestart to end of the exposure of at least one exposure line. As a result,an image with clear luminance is obtained as in 7503 d, from which theluminance change of the light source is easily recognizable. In the casewhere 2t_(E)>t_(S), a bright line pattern different from the luminancechange of the light source might be obtained, making it difficult torecognize the luminance change of the light source from the capturedimage.

FIG. 57F illustrates the relation between the transition time t_(T) oflight source luminance and the time difference t_(D) between theexposure start times of the exposure lines. When t_(D) is large ascompared with t_(T), fewer exposure lines are in the intermediate color,which facilitates estimation of light source luminance. It is desirablethat t_(D)>t_(T), because the number of exposure lines in theintermediate color is two or less consecutively. Since t_(T) is lessthan or equal to 1 microsecond in the case where the light source is anLED and about 5 microseconds in the case where the light source is anorganic EL device, setting t_(D) to greater than or equal to 5microseconds facilitates estimation of light source luminance.

FIG. 57G illustrates the relation between the high frequency noiset_(HT) of light source luminance and the exposure time t_(E). When t_(E)is large as compared with t_(HT), the captured image is less influencedby high frequency noise, which facilitates estimation of light sourceluminance. When t_(E) is an integral multiple of t_(HT), there is noinfluence of high frequency noise, and estimation of light sourceluminance is easiest. For estimation of light source luminance, it isdesirable that t_(E)>t_(HT). High frequency noise is mainly caused by aswitching power supply circuit. Since t_(HT) is less than or equal to 20microseconds in many switching power supplies for lightings, settingt_(E) to greater than or equal to 20 microseconds facilitates estimationof light source luminance.

FIG. 57H is a graph representing the relation between the exposure timet_(E) and the magnitude of high frequency noise when t_(HT) is 20microseconds. Given that t_(HT) varies depending on the light source,the graph demonstrates that it is efficient to set t_(E) to greater thanor equal to 15 microseconds, greater than or equal to 35 microseconds,greater than or equal to 54 microseconds, or greater than or equal to 74microseconds, each of which is a value equal to the value when theamount of noise is at the maximum. Though t_(E) is desirably larger interms of high frequency noise reduction, there is also theabove-mentioned property that, when t_(E) is smaller, anintermediate-color part is less likely to occur and estimation of lightsource luminance is easier. Therefore, t_(E) may be set to greater thanor equal to 15 microseconds when the light source luminance changeperiod is 15 to 35 microseconds, to greater than or equal to 35microseconds when the light source luminance change period is 35 to 54microseconds, to greater than or equal to 54 microseconds when the lightsource luminance change period is 54 to 74 microseconds, and to greaterthan or equal to 74 microseconds when the light source luminance changeperiod is greater than or equal to 74 microseconds.

FIG. 57I illustrates the relation between the exposure time t_(E) andthe recognition success rate. Since the exposure time t_(E) is relativeto the time during which the light source luminance is constant, thehorizontal axis represents the value (relative exposure time) obtainedby dividing the light source luminance change period t_(S) by theexposure time t_(E). It can be understood from the graph that therecognition success rate of approximately 100% can be attained bysetting the relative exposure time to less than or equal to 1.2. Forexample, the exposure time may be set to less than or equal toapproximately 0.83 millisecond in the case where the transmission signalis 1 kHz. Likewise, the recognition success rate greater than or equalto 95% can be attained by setting the relative exposure time to lessthan or equal to 1.25, and the recognition success rate greater than orequal to 80% can be attained by setting the relative exposure time toless than or equal to 1.4. Moreover, since the recognition success ratesharply decreases when the relative exposure time is about 1.5 andbecomes roughly 0% when the relative exposure time is 1.6, it isnecessary to set the relative exposure time not to exceed 1.5. After therecognition rate becomes 0% at 7507 c, it increases again at 7507 d,7507 e, and 7507 f. Accordingly, for example to capture a bright imagewith a longer exposure time, the exposure time may be set so that therelative exposure time is 1.9 to 2.2, 2.4 to 2.6, or 2.8 to 3.0. Such anexposure time may be used, for instance, as an intermediate mode in FIG.59.

Depending on imaging devices, there is a time (blanking) during which noexposure is performed, as illustrated in FIG. 58.

In the case where there is blanking, the luminance of the light emittingunit during the time cannot be observed.

A transmission loss caused by blanking can be prevented by the lightemitting unit repeatedly transmitting the same signal two or more timesor adding error correcting code.

To prevent the same signal from being transmitted during blanking everytime, the light emitting unit transmits the signal in a period that isrelatively prime to the period of image capture or a period that isshorter than the period of image capture.

Embodiment 6

FIG. 60 is a diagram illustrating a service provision system using thereception method described in any of the foregoing embodiments.

First, a company A ex8000 managing a server ex8002 is requested todistribute information to a mobile terminal, by another company B orindividual ex8001. For example, the distribution of detailedadvertisement information, coupon information, map information, or thelike to the mobile terminal that performs visible light communicationwith a signage is requested. The company A ex8000 managing the servermanages information distributed to the mobile terminal in associationwith arbitrary ID information. A mobile terminal ex8003 obtains IDinformation from a subject ex8004 by visible light communication, andtransmits the obtained ID information to the server ex8002. The serverex8002 transmits the information corresponding to the ID information tothe mobile terminal, and counts the number of times the informationcorresponding to the ID information is transmitted. The company A ex8000managing the server charges the fee corresponding to the count, to therequesting company B or individual ex8001. For example, a larger fee ischarged when the count is larger.

FIG. 61 is a flowchart illustrating service provision flow.

In Step ex8000, the company A managing the server receives the requestfor information distribution from another company B. In Step ex8001, theinformation requested to be distributed is managed in association withthe specific ID information in the server managed by the company A. InStep ex8002, the mobile terminal receives the specific ID informationfrom the subject by visible light communication, and transmits it to theserver managed by the company A. The visible light communication methodhas already been described in detail in the other embodiments, and soits description is omitted here. The server transmits the informationcorresponding to the specific ID information received from the mobileterminal, to the mobile terminal. In Step ex8003, the number of timesthe information is distributed is counted in the server. Lastly, in Stepex8004, the fee corresponding to the information distribution count ischarged to the company B. By such charging according to the count, theappropriate fee corresponding to the advertising effect of theinformation distribution can be charged to the company B.

FIG. 62 is a flowchart illustrating service provision in anotherexample. The description of the same steps as those in FIG. 61 isomitted here.

In Step ex8008, whether or not a predetermined time has elapsed from thestart of the information distribution is determined. In the case ofdetermining that the predetermined time has not elapsed, no fee ischarged to the company B in Step ex8011. In the case of determining thatthe predetermined time has elapsed, the number of times the informationis distributed is counted in Step ex8009. In Step ex8010, the feecorresponding to the information distribution count is charged to thecompany B. Since the information distribution is performed free ofcharge within the predetermined time, the company B can receive theaccounting service after checking the advertising effect and the like.

FIG. 63 is a flowchart illustrating service provision in anotherexample. The description of the same steps as those in FIG. 62 isomitted here.

In Step ex8014, the number of times the information is distributed iscounted. In the case of determining that the predetermined time has notelapsed from the start of the information distribution in Step ex8015,no fee is charged in Step ex8016. In the case of determining that thepredetermined time has elapsed, on the other hand, whether or not thenumber of times the information is distributed is greater than or equalto a predetermined number is determined in Step ex8017. In the casewhere the number of times the information is distributed is less thanthe predetermined number, the count is reset, and the number of timesthe information is distributed is counted again. In this case, no fee ischarged to the company B regarding the predetermined time during whichthe number of times the information is distributed is less than thepredetermined number. In the case where the count is greater than orequal to the predetermined number in Step ex8017, the count is reset andstarted again in Step ex8018. In Step ex8019, the fee corresponding tothe count is charged to the company B. Thus, in the case where the countduring the free distribution time is small, the free distribution timeis provided again. This enables the company B to receive the accountingservice at an appropriate time. Moreover, in the case where the count issmall, the company A can analyze the information and, for example whenthe information is out of season, suggest the change of the informationto the company B. In the case where the free distribution time isprovided again, the time may be shorter than the predetermined timeprovided first. The shorter time than the predetermined time providedfirst reduces the burden on the company A. Further, the freedistribution time may be provided again after a fixed time period. Forinstance, if the information is influenced by seasonality, the freedistribution time is provided again after the fixed time period untilthe new season begins.

Note that the charge fee may be changed according to the amount of data,regardless of the number of times the information is distributed.Distribution of a predetermined amount of data or more may be charged,while distribution is free of charge within the predetermined amount ofdata. The charge fee may be increased with the increase of the amount ofdata. Moreover, when managing the information in association with thespecific ID information, a management fee may be charged. By chargingthe management fee, it is possible to determine the fee upon requestingthe information distribution.

The above is a description of an information communication method and aninformation communication device according to one or more aspects of thepresent disclosure based on the embodiments and variations thereof. Thepresent disclosure, however, is not limited to the embodiments and thevariations. Various modifications to the embodiments that may beconceived by those skilled in the art and combinations of constituentelements in different embodiments may be included within the scope ofthe one or more aspects of the present disclosure, without departingfrom the spirit of the present disclosure.

It should be noted that the following cases are also encompassed in thepresent disclosure.

(1) Specifically, each device described above may be a computer systemwhich includes a microprocessor, a ROM, a RAM, a hard disk unit, adisplay unit, a keyboard, a mouse, and the like. A computer program isstored in the RAM or the hard disk unit. The operation of themicroprocessor in accordance with the computer program allows eachdevice to achieve its functionality. Here, the computer program includesa combination of instruction codes indicating instructions to a computerin order to achieve given functionality.

(2) Some or all of constituent elements included in each devicedescribed above may include a single system large scale integration(LSI: large scale integrated circuit). The system LSI is a supermulti-function LSI manufactured by integrating plural components in onechip, and is specifically a computer system configured so as to includea microprocessor, a ROM, a RAM, and so on. A computer program is storedin the RAM. The system LSI accomplishes its functions through theoperation of the microprocessor in accordance with the computer program.

(3) Some or all of constituent elements included in each devicedescribed above may include an IC card or a single module which can beattached to or detached from the device. The IC card or the module is acomputer system which includes a microprocessor, a ROM, a RAM, and thelike. The above super-multifunctional LSI may be included in the IC cardor the module. The IC card or the module accomplishes its functionsthrough the operation of the microprocessor in accordance with thecomputer program. This IC card or module may have tamper resistantproperties.

(4) The present disclosure may be achieved by the methods describedabove. In addition, these methods may be achieved by a computer programimplemented by a computer, or may be implemented by a digital signalwhich includes the computer program.

The present disclosure may be achieved by the computer program or thedigital signal stored in a computer-readable recording medium such as,for example, a flexible disk, a hard disk, CD-ROM, MO, DVD, DVD-ROM,DVD-RAM, a blu-ray Disc (BD), or a semiconductor memory. Alternatively,the present disclosure may be achieved by the digital signal stored insuch a recording medium.

According to the present disclosure, the computer program or the digitalsignal may be transmitted via, for instance, data broadcasting or anetwork typified by electric telecommunication lines, wireless or wiredcommunication lines, and the Internet. The present disclosure may be acomputer system which includes a microprocessor and a memory, the memorymay have stored therein the computer program, and the microprocessor mayoperate in accordance with the computer program.

Another independent computer system may implement the program or thedigital signal which has been delivered thereto being stored in therecording medium or the program or the digital signal delivered theretovia the network or the like.

(5) The above embodiments and the above variations may be combined.

Although the above is a description of exemplary embodiments, the scopeof the claims of the present application is not limited to thoseembodiments. Without departing from novel teaching and advantages of asubject matter described in the appended claims, various modificationsmay be made to the above embodiments, and constituent elements in theabove embodiments may be arbitrarily combined to achieve anotherembodiment, which is understood with ease by a person skilled in theart. Therefore, such modifications and other embodiments are alsoincluded in the present disclosure.

INDUSTRIAL APPLICABILITY

The present disclosure can be used for information communication devicesand others, and in particular for information communication devices, forinstance, which are used for a method for communication between mobileterminals and home electric appliances, the mobile terminals including asmartphone, a tablet terminal, a mobile phone, a smart watch, and a headmount display, the home electric appliances including anair-conditioner, a lighting device, a rice cooker, a television, arecorder, and a projector.

We claim:
 1. An information communication method, comprising: continuously capturing an image of a subject that transmits a signal by changing luminance, with an image sensor; displaying a captured image that includes a box and a subject image for identifying the subject, while continuously updating a position of the subject image on the captured image, according to a positional relationship between the image sensor and the subject established when the image of the subject is captured; determining whether the subject image is in the box; and receiving the signal transmitted by the subject when it is determined that the subject image is in the box, wherein in the receiving, second image data is obtained by performing capturing with a second exposure time shorter than a first exposure time so that exposure starts sequentially for a plurality of exposure lines each at a different time and each of the plurality of exposure lines partially overlaps in exposure time an adjacent one of the plurality of exposure lines, and information is obtained by demodulating a bright line pattern that appears in the second image data and corresponds to the plurality of exposure lines, and the bright line pattern is caused to appear in the second image data by setting the second exposure time to be less than or equal to 1/480 second.
 2. The information communication method according to claim 1, further comprising: determining whether a terminal device that includes the image sensor has been shaken laterally, wherein in the capturing, images of plural subjects each of which transmits a signal by changing luminance are captured with the image sensor, in the displaying, a captured image is displayed, the captured image including the images of the plural subjects and being obtained in the capturing, in the receiving, the signals transmitted by the plural subjects are received, and in the displaying, an information notification image generated according to a signal from one of the plural subjects among the signals transmitted by the plural subjects and received, and showing information regarding the subject is superimposed on the captured image in a state where the information notification image is associated with the image of the subject, and when it is determined that the terminal device has been shaken laterally, another information notification image generated according to a signal from another one of the plural subjects that is in a direction in which the terminal device has been shaken, and showing information regarding the another one of the plural subjects is superimposed on the captured image in a state where the other information notification image is associated with the image of the another one of the plural subjects.
 3. The information communication method according to claim 2, wherein in the displaying, for an image of each of one or more remaining subjects other than the image of the one or the another one of the plural subjects that is associated with the information notification image or the another information notification image, an information-presentable notification image for notifying that information regarding a remaining subject is further presentable is superimposed on the captured image.
 4. An information communication apparatus, comprising: a processor; and a memory having thereon a computer program, the computer program causing the processor to execute operations including: continuously capturing an image of a subject that transmits a signal by changing luminance, with an image sensor; displaying a captured image that includes a box and a subject image for identifying the subject, while continuously updating a position of the subject image on the captured image, according to a positional relationship between the image sensor and the subject established when the image of the subject is captured; determining whether the subject image is in the box; and receiving the signal transmitted by the subject when it is determined that the subject image is in the box, wherein in the receiving, second image data is obtained by performing capturing with a second exposure time shorter than a first exposure time so that exposure starts sequentially for a plurality of exposure lines each at a different time and each of the plurality of exposure lines partially overlaps in exposure time an adjacent one of the plurality of exposure lines, and information is obtained by demodulating a bright line pattern that appears in the second image data and corresponds to the plurality of exposure lines, and the bright line pattern is caused to appear in the second image data by setting the second exposure time to be less than or equal to 1/480 second.
 5. A non-transitory computer-readable recording medium storing an information communication program for performing information communication, the information communication program causing a computer to execute: continuously capturing an image of a subject that transmits a signal by changing luminance, with an image sensor; displaying a captured image that includes a box and a subject image for identifying the subject, while continuously updating a position of the subject image on the captured image, according to a positional relationship between the image sensor and the subject established when the image of the subject is captured; determining whether the subject image is in the box; and receiving the signal transmitted by the subject when it is determined that the subject image is in the box, wherein in the receiving, second image data is obtained by performing capturing with a second exposure time shorter than a first exposure time so that exposure starts sequentially for a plurality of exposure lines each at a different time and each of the plurality of exposure lines partially overlaps in exposure time an adjacent one of the plurality of exposure lines, and information is obtained by demodulating a bright line pattern that appears in the second image data and corresponds to the plurality of exposure lines, and the bright line pattern is caused to appear in the second image data by setting the second exposure time to be less than or equal to 1/480 second. 